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PREFACE 

This  monograph  is  the  fourth  in  a  series  of  technical  reports  presenting 
findings  of  the  Consumer  Savings  Project  of  the  Inter-University  Com- 
mittee for  Research  on  Consumer  Behavior.  The  first  and  second  publica- 
tions in  this  series  presented  methodological  results  of  the  project,  while 
the  third  publication  related  to  consumer  savings  behavior. 

This  fourth  monograph  presents  some  further  methodological  findings, 
focusing  on  the  role  of  the  interviewer  in  affecting  the  quality  of  data 
obtainable  on  personal  interview  surveys.  The  validation  of  financial 
information  reported  by  the  respondents  represented  a  unique  opportunity 
to  evaluate  interviewer  effectiveness  in  obtaining  accurate  and  complete 
financial  data.  The  selection,  training,  and  supervision  procedures  em- 
ployed on  the  project  studies  are  described  and  evaluated  in  this  mono- 
graph, with  the  validation  information  being  used  to  provide  objective 
measures  of  interviewer  effectiveness.  On  the  basis  of  these  findings,  Dr. 
Steinkamp  and  Mr.  Hauck  present  some  thought-provoking  hypotheses 
on  procedures  which  may  help  improve  data  on  future  financial  surveys. 
This  project  has  been  financed  by  a  grant  from  the  Ford  Foundation, 
with  supplemental  assistance  from  the  United  States  Department  of  Agri- 
culture. Financial  support  for  additional  work  currently  under  way  has 
been  provided  by  the  National  Science  Foundation  and  by  the  United 
States  Department  of  Labor.  Robert  Ferber,  Research  Professor  of 
Economics  at  the  University  of  Illinois,  is  director  of  the  project. 

The  members  of  the  Inter-University  Committee  for  Research  on 
Consumer  Behavior  are: 

Lincoln  Clark,  New  York  University,  Secretary-Treasurer 

Robert  Ferber,  University  of  Illinois 

George  Katona,  University  of  Michigan 

Theodore  Newcomb,  University  of  Michigan 

Howard  Raiffa,  Harvard  University 


James  Tobin,  Yale  University 
Guy  Orcutt,  University  of  Wisconsin,  Chairman 
Raymond  Goldsmith  was  a  member  of  the  Committee  until  he  left  in 
June,  1963,  for  an  OECD  assignment  in  Paris. 

The  monographs  in  this  series  are  research  reports.  The  Inter-Univer- 
sity Committee,  as  sponsor  of  this  research,  makes  every  effort  to  ensure 
both  the  quality  of  the  reports  and  their  orientation  toward  meeting  a 
real  need.  Nevertheless,  the  findings  reported  in  this  way  summarize 
conclusions  arrived  at  by  project  staff  and  do  not  necessarily  represent  the 
individual  or  collective  views  of  the  members  of  the  Inter-University 
Committee. 

Guy  Orcutt,  Chairman 

Inter-University  Committee 

for  Research  on  Consumer  Behavior 
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I.  BACKGROUND  AND  INTRODUCTION 


The  survey  method  is  unique  in  its  ability  to  obtain  information  con- 
cerning individual  decision-making  units.  The  variety  of  information 
obtained  about  each  unit  permits  the  study  of  relationships  between 
different  aspects  of  financial  behavior.  Thus,  information  about  income, 
assets,  debts,  savings,  and  the  like  permits  generalizations  concerning  the 
economic  behavior  of  the  whole  population  or  of  groups  in  the  popula- 
tion. Knowledge  so  gained  gives  insights  into  the  nature  of  aggregate 
relationships  and  forms  the  basis  for  simulations  of  the  economy.  Yet 
this  advantage  of  survey  methods  may  be  largely  nullified  if  the  informa- 
tion obtained  is  not  accurate  and  reliable. 

One  approach  to  evaluating  the  accuracy  and  reliability  of  survey 
data  is  the  comparison  of  aggregates  derived  from  surveys  with  those 
derived  independently  by  the  Departments  of  Commerce,  Labor,  and 
Agriculture  and  other  sources.  Such  comparisons  indicate  underreporting 
of  certain  holdings,  especially  liquid  assets  and  personal  debt.1  However, 
these  comparisons  frequently  involve  heroic  efforts  through  adjustments 
to  make  the  data  comparable,  although  they  offer  little  insight  into  the 
reasons  for  the  discrepancies. 

The  accuracy  and  completeness  of  sample  surveys  can  also  be  evalu- 
ated at  the  level  of  the  individual  sample  members.  Using  institutional 
records,  information  about  the  true  holdings  of  sample  members  can  be 
compared  with  the  information  obtained  at  the  sample  addresses.  This 
approach  gives  insights  into  the  reasons  for  bias  in  aggregates  prepared 
from  survey  data.  Such  bias  may  result  from  failure  to  obtain  an  inter- 
view at  the  address  of  the  sample  member  or  may  result  from  the  persons 
interviewed  giving  incomplete  or  inaccurate  information.  In  addition, 
bias  in  the  sample  variance,  which  affects  the  reliability  of  tests  of 
significance,  can  be  evaluated. 

The  Consumer  Savings  Project 

The  Consumer  Savings  Project  was  set  up  primarily  to  evaluate  the 
reliability  of  data  obtained  by  sample  surveys.    The  methodological  aims 

1  Robert   Ferber,   The  Measurement   and   Control  of  Errors:    Savings  Data, 
Chapter  3,  manuscript  in  preparation. 


2  Survey  Reliability  and  Interviewer  Competence 

of  the  project  were  first  to  evaluate  the  completeness  and  accuracy  of 
financial  survey  data,  and  second  to  develop  methods  of  improving  the 
completeness  and  accuracy  of  information  obtained  from  consumer  finan- 
cial surveys.  The  project  was  carried  out  by  means  of  a  series  of  panel 
operations  conducted  in  different  parts  of  the  country  and  at  different 
times. 

Some  of  these  operations  were  conducted  by  the  Survey  Research 
Center  of  the  University  of  Michigan,  and  others  were  conducted  under 
the  direct  supervision  of  the  project  staff  at  the  University  of  Illinois. 
Each  of  the  operations  had  specific  objectives  relating  to  some  aspect  of 
collecting  financial  data  from  consumers.  In  most  instances  the  accuracy 
and  completeness  of  the  survey  data  was  evaluated  in  terms  of  records 
made  available  through  the  cooperation  of  financial  institutions.  In 
many  of  the  studies  these  records  served  as  a  frame  for  sample  selection. 

The  reason  for  the  sequential  procedure  in  conducting  the  panel 
studies  was  to  permit  the  design  of  each  succeeding  operation  to  be 
modified  in  accordance  with  the  results  obtained  from  the  prior  studies. 
In  particular,  knowledge  of  the  results  of  the  experiments  in  one  study 
could  serve  as  the  basis  for  new,  and,  it  was  hoped,  more  useful  experi- 
ments in  a  later  study.  Results  obtained  regarding  the  accuracy  of  finan- 
cial survey  data  are  discussed  in  detail  elsewhere.2  A  brief  summary  of 
validation  results  for  time  deposits  in  one  study  is  sufficient  to  provide 
some  idea  of  the  magnitude  of  the  errors  which  occurred.  It  represents 
an  extreme  though  important  example  of  the  kinds  of  errors  which  may 
occur.  The  existence  of  the  validated  savings  account  was  ascertained  for 
about  60  percent  of  the  sample  addresses  where  contact  was  made.  In 
addition,  the  total  amount  reported  by  respondents  who  mentioned  vali- 
dated accounts  was  a  little  over  two-fifths  of  the  total  holdings  of 
persons  at  the  sample  addresses  where  contact  was  made.  Possibly  of 
greater  importance  was  the  finding  that  the  computed  sampling  variances 
were  biased  estimates  of  the  true  variances.3  As  a  result,  tests  of  signifi- 
cance had  limited  meaning.  Results  of  this  nature  serve  to  emphasize 
the  importance  of  the  second  methodological  objective  of  the  project, 
that  of  improving  the  accuracy  and  reliability  of  survey  data. 

The  accuracy  and  completeness  of  information  obtained  by  the  use  of 
survey  techniques  can  be  improved  through  two  interrelated  approaches. 
One  is  to  improve  survey  methods.    This  approach  emphasizes  improved 

2  See  J.  B.  Lansing,  G.  P.  Ginsburg,  and  Kaisa  Braaten,  An  Investigation  of 
Response  Error,  Studies  in  Consumer  Savings  No.  2  (Urbana:  University  of  Illi- 
nois, Bureau  of  Economic  and  Business  Research,  1961);  Robert  Ferber,  The 
Measurement  and  Control  of  Errors:    Savings  Data,  op.  cit. 

3  See  Robert  Ferber,  ibid. 
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methods  of  pre-interview  contact;  of  selecting,  training,  and  super- 
vising interviewers;  and  of  questionnaire  construction.  The  second  ap- 
proach is  to  develop  techniques  for  correcting  and  adjusting  information 
obtained  from  population  samples  and  validation  samples  in  terms  of 
each  other.4 

The  present  monograph  is  concerned  with  the  first  of  these  ap- 
proaches, specifically  with  interviewer  selection,  training,  and  supervision, 
and  with  the  extent  to  which  survey  reliability  appears  to  depend  on  the 
interviewer  variability.  In  the  latter  respect,  the  availability  of  validation 
data  presented  a  unique  opportunity  to  evaluate  interviewer  performance 
and  to  use  an  objective  measure  of  the  quality  of  information  obtained  in 
the  interview  as  a  basis  for  such  an  evaluation. 

The  Individual  Studies 

This  monograph  is  based  on  four  panel  studies  with  which  the  authors 
were  involved.  These  studies  constituted  a  major  part  of  the  over-all 
project.5 

In  each  of  the  four  studies,  the  respondents  were  interviewed  five 
times  at  regular  intervals  three  or  four  months  apart.  In  this  way,  prob- 
lems of  obtaining  savings  information  on  a  continuing  basis  could  also 
be  studied. 

The  unit  studied  was  the  savings  unit.  A  savings  unit  was  defined  as 
one  or  more  related  persons  living  in  the  same  dwelling  unit  who  pooled 
half  or  more  of  their  income  and  savings.  A  dwelling  might  therefore 
have  had  more  than  one  savings  unit.  Savings  of  minors  were  combined 
with  those  of  their  legal  guardians  whether  or  not  the  minors  had  separate 
savings.  This  definition  had  two  implications  for  the  data  collected. 
First,  the  savings  unit  was  a  decision  unit,  thus  facilitating  the  study  of 
saving  behavior.  Second,  the  head  of  the  savings  unit  or  some  member 
of  it  usually  had  complete  and  accurate  information  about  the  holdings 
of  the  unit  as  well  as  knowledge  about  changes  in  the  holdings  of  the 
savings  unit. 

The  studies  were  undertaken  in  four  different  areas  in  the  Midwest. 
The  first  study,  referred  to  as  Ml,  was  made  in  a  large  metropolitan 
center.  The  second  study  (M2)  was  undertaken  in  another  metropolitan 
center.  The  third  study  (M3)  was  conducted  in  a  smaller  urban  center, 
and  the  last  study  (M4)  was  conducted  in  three  farm  counties  in  the 
Midwest.   A  brief  summary  of  each  of  these  studies  follows. 

4  For  further  details  on  this  approach,  see  ibid. 

5  The  other  studies  were  carried  out  by  the  Survey  Research  Center,  University 
of  Michigan.   See  J.  B.  Lansing  et  al.,  op.  cit. 
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Study  Ml6 

Study  Ml  was  intended  to  test  the  feasibility  of  obtaining  income, 
savings,  and  other  financial  data  from  consumers  on  a  continuing  basis. 
More  specifically,  the  objectives  were  (1)  to  estimate  costs  and  response 
rates  on  consumer  financial  panel  operations,  (2)  to  gain  insights  into 
the  problems  of  obtaining  financial  data  on  a  continuing  basis  and  to 
explore  different  methods  of  doing  so,  and  (3)  to  explore  the  use  of 
internal  means  of  assessing  reliability  of  these  data. 

A  probability  sample  of  300  names  was  selected  from  the  population 
of  the  metropolitan  area  in  the  fall  of  1956.  Census  tracts  were  classified 
into  three  income  strata  based  on  data  from  the  1950  Census  Tracts 
volume  for  the  area.  A  systematic  selection  of  census  tracts  from  each 
stratum  was  made  with  a  disproportionately  high  number  of  tracts  from 
the  higher  income  strata.  Names  and  addresses  were  selected  from  each 
sample  tract  through  the  use  of  voter  registration  lists  and  telephone 
directories. 

Each  of  the  sample  families  was  sent  a  personal  letter  on  University 
of  Illinois  stationery  notifying  it  of  the  general  nature  of  the  survey  and 
the  forthcoming  call  by  a  named  interviewer.  Personal  advance  letters 
were  also  sent  at  the  beginning  of  each  succeeding  wave  of  interviews. 
The  letters  for  subsequent  waves  differed  for  some  respondents  since  they 
were  used  as  a  means  of  answering  questions  raised  during  the  interviews 
of  the  previous  wave. 

The  actual  interviewing  was  largely  subcontracted  to  two  survey  or- 
ganizations. For  purposes  of  obtaining  comparative  information,  some 
interviews  were  assigned  to  two  interviewers  hired  directly  by  the  project. 

Each  panel  member  was  asked  numerous  questions  about  his  financial 
situation,  about  record-keeping,  and  about  the  use  of  budgets.  In  addi- 
tion, one  segment  of  the  sample  was  asked  to  keep  records  or  diaries 
provided  by  the  project.  On  succeeding  interview  waves,  the  panel  mem- 
ber's financial  situation  was  reviewed  and  changes  in  it  were  recorded. 
The  interviews  were  conducted  with  the  head  of  the  savings  unit  or  with 
the  member  of  the  family  who  was  familiar  with  the  savings  unit's 
finances. 

Study  M2 

Study  M2  was  begun  in  mid- 1958  with  a  sample  of  308  names.  The 
sample  was  an  area  probability  sample  with  disproportionate  representa- 


0  For  complete  details  on  this  study,  see  Robert  Ferber,  Collecting  Financial 
Data  by  Consumer  Panel  Techniques,  Studies  in  Consumer  Saving  No.  1  (Urbana: 
University  of  Illinois,  Bureau  of  Economic  and  Business  Research,  1960). 
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tion  given  to  high-income  sample  members.  Over  40  percent  of  the 
savings  units  were  estimated  to  have  had  incomes  in  1959  exceeding 
$10,000;  the  corresponding  1950  Census  estimate  for  the  sample  area  was 
10  percent. 

Respondents  were  interviewed  a  total  of  five  times  over  the  period  of 
a  year  and  a  half.  The  respondent  for  each  savings  unit  was  asked  ques- 
tions about  all  the  assets  and  debts  of  the  savings  unit.  Included  were 
checking  accounts,  savings  accounts,  life  insurance  holdings,  annuities, 
pensions,  brokerage  accounts,  and  the  ownership  of  businesses.  The  field 
operation  was  conducted  by  interviewers  hired,  trained,  and  supervised 
by  a  field  director. 

A  number  of  experiments  relating  to  the  different  interview  ap- 
proaches were  conducted  in  the  course  of  the  five  waves  of  interviews. 
When  more  than  one  experiment  was  conducted  on  a  single  wave,  the 
sample  was  divided  by  systematic  selection  into  as  many  random  segments 
as  there  were  combinations  of  experiments  and  each  segment  received  a 
different  combination  —  a  factorial  design.  The  experiments  of  the  first 
wave  were  the  following,  with  the  response  rates  for  each  experiment  in 
parentheses.7 

1.  Use  of  a  short  letter  of  introduction  (76  percent)  versus  use  of  a 
long  detailed  letter  (90  percent). 

2.  Use  of  a  highly  structured  questionnaire  (72  percent)  versus  use  of 
no  questionnaire,  i.e.,  unstructured  interviews  (87  percent). 

3.  Request  for  dollar  amounts  in  the  interview  (77  percent)  versus 
no  request  for  dollar  amounts  (81  percent) . 

4.  Request  for  assets  first  in  the  interview  (72  percent)  versus  request 
for  debts  first  (73  percent). 

Two  types  of  financial  holdings  reported  by  the  persons  interviewed 
were  validated,  personal  debt  and  life  insurance  holdings.  Secondary-type 
validation  was  employed  in  each  case.  Holdings  reported  by  the  sample 
members  were  sent  to  cooperating  financial  institutions  who  reported  the 
correct  amount  held  by  the  sample  members  on  some  specified  date, 
usually  the  date  of  the  interview.  This  amount  was  then  used  to  evaluate 
the  accuracy  of  the  amount  reported  by  the  sample  member.  Secondary 
validations  provided  no  information  about  sample  addresses  where  no 
contact  was  made  or  where  the  respondent  refused  to  be  interviewed. 
In  addition,  they  provided  little  or  no  information  about  holdings  which 
interviewed  sample  members  failed  to  report. 

7  Further   details   on   the   experiments   are   presented   in   Robert   Ferber,   The 
Measurement  and  Control  of  Errors:    Savings  Data,  op.  cit.,  Chapter  13. 
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Study  M3 

In  interviews  undertaken  in  1959  in  a  smaller  urban  center,  panel 
members  in  Study  M3  were  interviewed  five  times  at  three-to-four-month 
intervals.  The  study  began  with  316  interviews  on  Wave  1  and  finished 
with  205  interviews  on  Wave  5.  The  following  split-run  experiments  were 
conducted  on  the  first  wave,  with  the  response  rates  for  each  experiment 
shown  in  parentheses.8 

1.  Use  of  a  regular  advance  letter  (81  percent)  versus  use  of  the 
same  letter  but  including  an  attempt  to  involve  the  panel  member  by 
seeking  his  suggestions  and  aid  in  the  conduct  of  the  interview  (81  per- 
cent) . 

2.  Use  of  a  questionnaire  form  on  which  answers  were  recorded  (77 
percent)  versus  use  of  an  unstructured  type  of  interview  with  reminder 
cards  to  aid  the  interviewer  (82  percent) . 

3.  Request  for  dollar  holdings  of  assets  and  debts  on  the  day  of  the 
interview  (73  percent)  versus  request  for  only  the  changes  that  had  taken 
place  over  the  past  three  months  (81  percent) . 

4.  Request  for  information  on  all  assets  and  debts  (80  percent)  versus 
request  for  information  on  specific  assets  only  (78  percent). 

Experiments  conducted  on  later  waves  included  the  following: 

1.  Rotation  of  interviewers  among  selected  panel  members  on  the 
third  wave. 

2.  Request  to  each  respondent  on  the  third  wave  to  sign  a  form 
authorizing  the  institution  to  report  the  current  balance  in  his  savings 
account. 

3.  Use  of  a  mail  questionnaire  sent  to  part  of  the  sample  on  the 
fourth  wave.  As  with  the  earlier  study,  a  factorial  design  was  employed. 

The  initial  sample  of  316  persons  was  selected  from  the  time  deposit 
records  of  savings  institutions  in  the  area.  When  an  interview  was  ob- 
tained, the  balance  on  the  day  of  the  interview  was  reported  to  the  project 
by  the  cooperating  institution.  For  a  sample  address  where  there  was  no 
contact  or  where  there  was  a  refusal,  a  midpoint  date  for  the  field  opera- 
tion was  selected  and  the  institution  reported  the  person's  balance  for  that 
day.  Interviewer  effectiveness  was  evaluated  in  terms  of  these  validation 
data. 

Two  secondary  validations  were  also  undertaken.  One  involved  life 
insurance  holdings.  As  in  the  earlier  study,  all  policies  were  validated 
when  the  respondent  mentioned  the  name  of  a  cooperating  institution. 
The  other  secondary  validation  involved  demand  deposits  mentioned  by 

8  Ibid. 
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respondents  at  two  of  the  financial  institutions.  Because  of  the  volatile 
nature  of  demand  deposits  and  because  of  the  existence  of  float  (checks 
which  have  not  cleared  the  institution),  a  balance  reported  by  the 
validating  institution  for  the  date  of  the  interview  could  not  be  directly 
comparable  with  that  reported  in  the  interview.  However,  corrections 
made  for  these  differences  indicated  a  fairly  good  correspondence  between 
reported  and  true  holdings. 

Study  M4  (The  Farm  Study) 

The  farm  study  was  begun  late  in  1960  in  three  counties  in  the  Mid- 
west. Half  of  the  roughly  400  farm  panel  members  were  interviewed 
three  times  at  about  six-month  intervals,  and  the  other  half  were  inter- 
viewed five  times  at  about  three-month  intervals.  Two  experiments  were 
conducted  on  the  first  wave  and  as  in  earlier  studies,  a  factorial  design 
was  used.9  These  experiments  and  the  response  rates  for  each  (in 
parenthesis)  were  as  follows: 

1.  Use  of  a  letter  of  introduction  which  made  a  straightforward 
appeal  based  on  the  need  for  information  about  how  farmers  handle 
their  finances  (90  percent)  versus  use  of  another  letter  with  the  same 
basic  approach  but  in  addition  an  appeal  to  sample  members  to  aid  in 
improving  the  data-collection  procedure,  as  in  Study  M3  (90  percent). 

2.  Use  of  one  questionnaire  requesting  the  farmers  to  report  their 
holdings  of  assets  and  debts,  including  the  farm  as  a  business,  on  the  day 
of  the  interview  (86  percent)  versus  another  questionnaire  requesting 
information  about  the  change  in  holdings  over  the  previous  three  months 
(93  percent) . 

In  this  study,  a  somewhat  more  extensive  pretest  was  used  to  refine 
the  questionnaires  and  to  evaluate  interviewer  performance.  The  sample 
was  selected  from  institutional  records  of  five  different  types  of  assets  and 
debts:  time  deposits,  demand  deposits,  short-term  debt,  intermediate- 
term  debt,  and  long-term  debt.  As  in  Study  M3,  names  selected  were 
checked  in  telephone  books  and  county  directories,  and  only  those  so 
located  were  used. 

It  was  found  that  cooperation  differed  substantially  between  sample 
members  selected  from  one  validation  source  and  those  selected  from 
other  sources.  As  a  result,  data  from  individuals  selected  from  different 
validation  sources  could  be  combined  only  to  a  limited  extent  in  the  same 
analysis.  This  severely  limited  the  number  of  comparable  contacts  made 
by  each  interviewer  and  accordingly  limits  the  scope  of  the  analysis  of 
these  data  presented  in  this  monograph. 

9  Ibid. 
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Organization  of  the  Monograph 

Knowledge  that  each  savings  unit  owned  a  certain  amount  of  a  spe- 
cific asset  or  debt  on  the  date  of  the  interview  provided  several  interrelated 
criteria  for  evaluating  interviewers.  Utilizing  such  data,  an  attempt  is 
made  here  to  understand  and  to  measure  the  relationship  between  inter- 
viewer effectiveness  and  techniques  of  interviewer  selection,  training,  and 
supervision.  In  Chapter  II  interviewer  variability  and  the  problems  of 
measuring  interviewer  performance  will  be  discussed.  Interviewer  selec- 
tion methods  will  be  presented  in  Chapter  III.  The  procedures  employed 
in  the  project  will  be  reviewed  and  evaluated,  followed  by  a  discussion 
of  a  quantitative  method  used  in  selecting  interviewers.  The  training  of 
interviewers  will  be  covered  in  Chapter  IV.  In  Chapter  V  interviewer 
supervision  and  the  controls  used  to  maintain  high  response  rates  with 
quality  data  will  be  discussed.  The  identification  of  effective  interviewers, 
based  on  their  interviewing  performance  when  no  validation  data  are 
available,  will  be  included  in  the  discussion. 

An  effort  will  be  made  throughout  the  monograph  to  identify  effec- 
tive interviewers,  a  problem  which  has  been  the  subject  of  much  pre- 
vious research.10  However,  a  direct  comparison  of  the  findings  of  such 
research  with  those  discussed  here  presents  one  or  more  of  the  following 
problems. 

1.  Much  of  the  earlier  research  was  concerned  with  interviewer  effec- 
tiveness in  attitudinal  studies.  In  such  studies,  interviewer  biasing  of 
respondent  answers  or  bias  in  recording  them  is  of  primary  importance. 
In  factual  studies  of  the  kind  reported  here,  this  type  of  bias  is  less  im- 
portant and  other  types  become  more  important.  In  the  absence  of 
tie-in  studies,  generalizing  from  one  to  the  other  type  of  study  must  be 
limited. 

2.  Earlier  studies  were  frequently  based  on  validation  information 
such  as  age,  voter  registration,  or  ownership  of  a  driver's  license,  about 
which  respondents  are  less  sensitive  than  about  financial  holdings.  The 
project  studies  were  based  primarily  on  validated  information  about  which 
respondents  are  often  highly  sensitive.  When  less  sensitive  information 
was  validated,  it  was  observed  that  interviewer  variability  was  much 
smaller.  This  suggests  that  factors  influencing  interviewer  effectiveness 
in  seeking  sensitive  data  may  not  be  so  important  when  less  sensitive 
data  are  sought. 

10  A  good  summary  of  past  research  is  presented  in  Herbert  H.  Hyman,  Inter- 
viewing in  Social  Research  (Chicago:  University  of  Chicago  Press,  1954).  Also, 
see  Robert  Ferber  and  H.  G.  Wales,  A  Basic  Bibliography  on  Marketing  Research, 
AM  A  Bibliography  Series  No.  2  (rev.)  (Chicago:  American  Marketing  Associa- 
tion, 1963),  Section  IX. 
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3.  The  criteria  of  interviewer  effectiveness  in  many  earlier  studies 
were  based  on  the  percentage  of  persons  contacted  who  refused  or  on  the 
number  of  times  information  was  not  ascertained,  refused,  or  reported  as 
not  known  in  the  interview.  The  present  analysis  indicates  that  these 
show  little  relationship  to  the  criteria  of  effectiveness  employed  here. 
Other  criteria  such  as  ratings  by  field  supervisors,  by  the  field  director,  or 
by  coders  similarly  show  little  correlation  with  the  measures  of  effective- 
ness employed.  A  criterion  employed  on  some  studies  is  interviewer 
variability  relative  to  the  average  variability  of  interviewers.  Such  analysis- 
of- variance  designs,  as  is  readily  admitted  by  those  using  them,11  do  not 
discriminate  between  good  and  poor  interviewers. 

4.  In  some  cases,  the  field  staffs  were  very  different  from  those  em- 
ployed in  the  project  research.  For  example,  Caplow  points  out  that  the 
field  staffs  used  in  many  of  the  earlier  studies  were  poorly  trained,  had 
high  turn-over  rates,  and  had  limited  supervision.12  The  method  of 
selecting  the  field  staff  may  also  have  influenced  findings.  If  the  inter- 
viewers selected  were  homogeneous  with  respect  to  some  characteristic, 
that  characteristic  as  well  as  related  characteristics  could  not  have  been 
analyzed. 

5.  Quota  sampling  was  frequently  employed  in  these  earlier  studies. 
As  a  result,  the  nature  of  the  sample  may  have  influenced  the  factors 
discovered  as  well  as  the  weight  attached  to  them. 

In  view  of  these  differences,  the  findings  of  earlier  studies  will  be 
reported  only  when  they  are  relevant  in  terms  of  these  five  dimensions. 

Since  1954  some  studies  have  been  made  with  which  the  project 
studies  may  be  directly  compared.  One  of  these  made  in  the  Netherlands 
compared  the  completeness  and  accuracy  with  which  savings  accounts 
were  reported  to  a  staff  of  interviewers.13  Another  made  by  the  Federal 
Reserve  Board  and  reported  by  Broida  described  the  accuracy  of  reports 
of  new  car  debt.14  However,  neither  of  these  studies  attempted  to  identify 
the  characteristics  of  the  more  effective  interviewers. 


11  For  example,  Robert  H.  Hanson  and  Eli  S.  Marks,  "Influence  of  the  Inter- 
viewer on  the  Accuracy  of  Survey  Results,"  Journal  of  American  Statistical  Associ- 
ation, Vol.  53,  No.  283  (September,  1958),  p.  648. 

12  Theodore  Caplow,  "The  Dynamics  of  Information  Interviewing,"  American 
Journal  of  Sociology,  Vol.  62,  No.  2  (September,  1956),  p.  167. 

13  W.  Horn,  "Reliability  Survey:  A  Survey  on  the  Reliability  of  Response  to 
an  Interview  Survey,"  Het  PTT-bedrijf,  Vol.  10,  No.  3  (October,  1960),  pp. 
105-56. 

14  Arthur  L.  Broida,  "Consumer  Surveys  as  a  Source  of  Information  for  Social 
Accounting:  The  Problems,"  in  Conference  on  Research  in  Income  and  Wealth, 
The  Flow-of-Funds  Approach  to  Social  Accounting:  Appraisals,  Analysis  and 
Applications,  Studies  in  Income  and  Wealth,  Vol.  26  (Princeton:  Princeton 
University  Press  for  the  National  Bureau  of  Economic  Research,  1962),  pp.  335-81. 
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The  findings  of  the  project  studies  made  by  the  Survey  Research 
Center  at  the  University  of  Michigan  relating  to  interviewers  and  of 
interaction  between  interviewers  and  respondents  are  integrated  into  the 
present  discussion.15 

15  J.  B.  Lansing,  et  al.,  op.  cit. 
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Measurement  of  Interviewer  Performance 

Criteria  of  interviewer  effectiveness  employed  in  surveys  for  which 
validation  data  are  not  available  include  the  contact  rate  (percentage  of 
eligible  assignments  contacted),  the  response  rate  (percentage  of  con- 
tacts interviewed),  measures  based  on  ambiguous  answers  in  the  ques- 
tionnaire, or  measures  of  promptness  in  completing  assignments,  coopera- 
tiveness,  and  related  factors.  These  criteria,  while  they  may  reflect  failure 
to  obtain  an  interview  and  failure  to  obtain  meaningful  answers,  do  not 
reflect  respondent  failure  to  mention  holdings  or  the  accuracy  with  which 
they  give  the  amounts  of  their  reported  holdings.  The  latter,  however, 
may  introduce  as  serious  a  bias  into  survey  data  as  the  former. 

Interviewers  may  also  be  evaluated  by  using  an  analysis  of  variance 
method  in  which  the  variation  in  results  obtained  by  different  interviewers 
is  compared  with  the  variation  between  respondents  queried  by  the  same 
interviewer.16  However,  with  this  approach  "it  is  not  possible  to  state 
that  one  result  represents  'good'  performance  and  another  result  'bad' 
performance.  We  can  state  that  Interviewer  A's  performance  differed 
from  that  of  Interviewer  B,  but  we  do  not  know  whether  the  direction 
of  difference  was  towards  'better'  or  towards  'worse'."17  In  addition,  this 
approach  reflects  only  the  differences  in  the  bias  of  interviewers  without 
saying  anything  about  the  amount  of  bias.  For  example,  "between  inter- 
viewer variance"  may  be  no  larger  than  "within  interviewer  variance" 
where  all  interviewers  introduce  the  same,  but  large,  amount  of  bias. 
Without  validation  information  about  the  true  characteristics  of  the 
respondents,  the  amount  of  bias  cannot  be  known. 

In  this  chapter,  criteria  of  interviewer  effectiveness  based  on  valida- 
tion data  are  defined  and  discussed.  Using  these  criteria,  interviewer 
differences  are  measured. 

In  the  present  analysis,  an  interviewer's  assignment  will  be  defined 
as  all  persons  who  maintain  their  home  at  the  sample  address.  Since  the 
problem  of  replacing  movers  in  the  sample  is  a  sampling  problem,  this 

10  For  example,  see  Robert  H.  Hanson  and  Eli  S.  Marks,  loc.  cit.,  pp.  635-55. 
17  Ibid.,  p.  648. 
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definition  of  assignments  is  the  one  applicable  for  evaluating  interviewer 
effectiveness. 

The  interviewer  may  have  failed  to  obtain  information  about  a  sample 
member  for  a  number  of  different  reasons.  First,  he  may  not  have  made 
contact  with  a  responsible  person  at  the  sample  address  because  some 
people  repeatedly  sent  children  or  servants  to  the  door  to  say  that  "no 
one  else  is  at  home."18  A  more  likely  situation  was  that  the  sample  mem- 
ber was  away  from  home  at  the  times  the  interviewer  could  make  a  call, 
especially  when  the  interviewer  was  not  allowed  to  telephone  for  appoint- 
ments as  was  the  case  on  the  first  wave  of  the  various  studies.  Clearly  the 
one  situation  did  not  reflect  interviewer  effectiveness,  though  the  other 
may  have  indicated  insufficient  effort  to  contact  the  sample  member  at 
different  times  of  the  day. 

Second,  the  interviewer  may  have  made  contact  with  a  responsible 
member  of  the  savings  unit  but  may  have  failed  to  obtain  information 
about  the  savings  unit  because  he  was  refused  an  interview.  Here,  inter- 
viewer performance  became  important.  It  was  the  interviewer's  job  to 
explain  the  nature  of  the  study,  its  purpose,  its  importance,  and  the 
confidential  treatment  of  the  data  and  to  answer  any  questions  which 
might  arise. 

Third,  although  an  interview  was  obtained,  the  information  elicited 
from  the  respondent  may  have  been  incomplete.  This  may  have  occurred 
because  the  respondent  forgot  about  a  holding,  because  he  did  not 
understand  what  information  was  wanted,  or  because  he  wished  to  con- 
ceal the  existence  of  a  holding.  In  each  case,  the  interviewer's  perform- 
ance was  likely  to  be  a  relevant  factor.  In  the  first  instance,  the 
interviewer  may  have  been  hurrying  the  respondent  or  not  using  sufficient 
probes.  In  the  second  instance,  the  interviewer  may  not  have  been  ask- 
ing the  questions  in  such  a  way  that  the  respondent  understood  what  was 
wanted.  In  the  last  case,  numerous  factors  relating  to  interviewer  tech- 
nique and  personality  may  have  explained  the  respondent's  withholding 
of  information. 

Fourth,  even  where  the  holding  was  reported  there  may  have  been  an 
error  in  the  amount  reported.  Inaccuracy  of  report  may  have  occurred 
because  of  memory  error  or  because  of  intentional  distortion.  Both  of 
these  were  subject  to  interviewer  influences.  Memory  error  could  have 
been  eliminated  through  the  use  of  records.  Even  when  records  were  not 
used,  the  accuracy  of  recall  was  likely  to  be  a  function  of  the  motivation 
of  the  respondent. 

18  An  excellent  discussion  on  this  is  to  be  found  in  H.  Lawrence  Koss,  "The 
Inaccessible  Respondent:  A  Note  on  Privacy  in  City  and  Country,"  Public  Opinion 
Quarterly,  Vol.  27,  No.  2  (Summer,  1963),  pp.  269-75. 
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Interviewer  effectiveness  in  such  situations  as  these  four  depends 
somewhat  on  different  aspects  of  interviewer  performance.  The  following 
criteria  relate  to  these  aspects. 

1.  Contact  rate  (K).  The  percentage  of  eligible  assignments  in  which 
the  interviewer  makes  contact  with  a  responsible  person.  The  inter- 
viewer's persistence  in  making  a  contact  is  measured  by  this  rate. 

2.  Response    rate    (R).     The    percentage    of    times    an    interviewer 

obtains  interviews  at  sample  addresses  where  contacts  are  made,   i.e.. 

Number  of  interviews     „ .  ,        •  ,  • 

rrz : ? .    Since  a  contact  must  be  either  an  interview  or  a 

Number  of  contacts 

r      .     .  .      .  ,        t        Number  of  refusals    ,_,  . 

refusal,  the  response  rate  is  also  equal  to  1  —  — = - .    Ihis 

Number  of  contacts 

rate  reflects  interviewer  effectiveness  at  the  door. 

3.  Completeness  rate  (C).  The  completeness  rate  is  the  percentage  of 
interviews  in  which  the  validated  holding  is  mentioned  by  the  respondent. 
This  rate  reflects  in  part  interviewer  effectiveness  in  the  interview.  How- 
ever, it  is  not  independent  of  the  response  rate.  A  low  response  rate  may 
imply  that  the  respondents  interviewed  are  more  likely  to  be  cooperative 
than  is  the  case  with  a  high  response  rate.  In  a  sense,  the  interviewer  with 
a  low  response  rate  can  be  thought  of  as  disposing  of  his  uncooperative 
sample  members  at  the  door  and  interviewing  only  the  relatively  coopera- 
tively ones,  therefore  obtaining  a  higher  completeness  rate. 

4.  Accuracy  rate  (A).  For  a  savings  account  that  is  reported,  this 
rate  measures  the  accuracy  with  which  the  amount  is  reported,  arbi- 
trarily defined  as  the  percentage  of  respondents  who  cite  amounts  within 
plus  or  minus  5  percent  of  the  true  amount.  A  range  of  10  percentage 
points  was  selected  because  even  among  the  most  cooperative  project 
respondents  there  was  a  tendency  to  fail  to  report  interest  payments  which 
had  not  been  added  to  the  passbook. 

When  change  information  is  requested,  the  accuracy  rate  is  defined 
as  the  percentage  of  respondents  reporting  change  in  the  balance  of  the 
account  within  plus  or  minus  25  percent  of  the  true  change.  The  use  of 
different  standards  for  change  and  holdings  data  reflects  the  fact  that 
change  is  generally  of  a  smaller  magnitude  than  holdings  and  thus  is 
likely  to  yield  a  larger  accuracy  ratio  for  a  given  amount  of  error.19  The 
accuracy  rate  is  dependent  on  the  response  and  the  completeness  rates. 
The  interviewer  with  low  response  and  completeness  rates  is  likely  to 

1S  An  alternative  to  using  the  accuracy  rate  for  evaluating  interviewer  effective- 
ness in  obtaining  accurate  data  would  be  the  average  accuracy  ratio,  based  on 
cumulation  of  dollar  totals.  However,  it  is  not  employed  because  of  the  dispropor- 
tionate impact  on  the  average  of  a  single  wealthy  respondent  who  reports  inac- 
curately. 
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have  only  the  most  cooperative  respondents  left  to  report  on  the  amount 
of  their  holdings  or  change.  As  a  result,  on  a  priori  grounds,  one  would 
expect  higher  levels  of  accuracy. 

From  these  basic  rates  other  rates  can  be  computed.  One  over-all 
criterion  of  interviewer  effectiveness  could  be  the  product  of  all  four  of 
these  measures:  K  X  R  X  C  X  A.  This  rate  reflects  the  percentage  of 
times  an  interviewer  obtains,  relative  to  the  number  of  his  total  eligible 
assignments,  reports  of  the  amounts  of  the  validated  accounts  to  within 
plus  or  minus  5  percent  of  the  true  amount. 

The  over-all  criterion  actually  selected  was  determined  by  the  nature 
of  the  studies.  A  policy  was  employed  of  requiring  each  interviewer  to 
make  at  least  five  attempts  to  contact  a  sample  member  before  turning 
in  a  noncontact  report.  When  an  interviewer  was  unable  to  make  contact 
the  sample  member  was  reassigned  to  a  different  interviewer.  In  Study 
M3,  on  all  except  one  reassignment,  the  second  interviewer  was  also 
unable  to  make  contact.  Analysis  of  the  13  noncontacts  in  this  study 
revealed  that  they  were  evenly  distributed  among  most  of  the  interviewers. 
For  these  reasons,  it  was  felt  that  noncontacts  did  not  reflect  interviewer 
ineffectiveness,  and  the  contact  rate  should  not  be  included  in  the  cri- 
terion of  effectiveness.  Hence,  the  over-all  measure  of  interviewer  effec- 
tiveness was  based  on  sample  addresses  where  contact  was  made  with  a 
responsible  person. 

An  experiment  undertaken  in  Study  M3  assigned  "change"  question- 
naires to  half  of  the  sample  and  "holding"  questionnaires  to  the  other 
half.  Subsequent  analysis  revealed  that  errors  in  reporting  the  amount 
of  change  were  not  comparable  with  errors  in  reporting  amount  of  the 
holding  on  either  an  absolute  or  a  relative  basis.  As  a  result,  a  criterion 
which  included  the  accuracy  rate  could  be  applied  to  one  or  the  other 
half  of  the  sample  at  a  time,  but  not  to  the  whole  sample.  In  view  of  the 
small  average  number  of  contacts  (17)  made  by  each  interviewer,  it  was 
decided  to  exclude  the  accuracy  rate  from  the  criterion  of  over-all  effec- 
tiveness. When  the  contact  rate  and  the  accuracy  rate  are  excluded,  the 
criterion  of  the  interviewer  effectiveness  employed  in  this  analysis  is  the 
pick-up  rate  (P)  which  is  the  percentage  of  contacts  in  which  the  in- 
terviewer picked  up  the  validated  account  (the  product  of  the  response 
rate  and  the  completeness  rate,  i.e.,  P  —  R  X  C) . 

It  should  be  stressed  that  the  value  of  this  or  related  criteria  depends 
on  the  accuracy  of  the  information  obtained  from  the  validation  source, 
which  is  not  necessarily  assured.  Inaccuracies  may  have  occurred  be- 
cause an  employee  at  the  validating  institution  recorded  the  balance 
for  the  wrong  date  or  recorded  it  incorrectly  for  the  correct  date.  A 
second  type  of  error  may  have  occurred  in  matching  accounts  reported 
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by  respondents  with  those  reported  by  the  institution.  To  reduce  this 
possibility,  the  names  of  all  financial  institutions  with  whom  the  respond- 
ent had  assets  or  debts  were  requested  in  the  interview.  The  impression 
gained  from  working  with  the  validation  process  was  that  very  few  errors 
could  be  attributed  to  the  validating  institution  or  to  the  matching 
process.  Moreover,  errors  discovered  in  the  validation  procedure  on 
subsequent  waves  were  corrected  in  the  data  for  earlier  waves  and  any 
completed  analysis  was  rerun. 

Problems  of  Generalization 

Interviewer  effectiveness  in  eliciting  information  about  one  char- 
acteristic may  not  be  indicative  of  effectiveness  in  obtaining  information 
about  other  characteristics.  For  example,  interviewers  who  are  effective 
in  eliciting  information  about  the  size  of  the  family  may  be  ineffective 
in  obtaining  data  about  the  family's  savings  accounts.  In  generalizing 
from  a  characteristic  which  was  validated  to  other  characteristics  which 
were  not  validated,  the  sensitivity  of  people  to  different  characteristics 
must  be  considered. 

Nor  is  sensitivity  a  unidimensional  concept.  People's  sensitivity  to 
reporting  information  may  vary  from  one  characteristic  to  another.  It 
may  also  vary  by  the  kind  of  information  requested.  For  example,  people 
appear  to  be  less  sensitive  about  reporting  a  change  in  balance  than  they 
are  about  reporting  an  actual  balance.  They  also  appear  to  be  more 
sensitive  about  reporting  a  large  holding  than  about  reporting  a  small 
holding.  Finally,  the  apparent  sensitivity  of  persons  in  reporting  a 
characteristic  depends  on  the  distribution  of  the  characteristic  in  the 
population.  For  example,  people  with  given  wealth  or  income  may  be 
less  sensitive  about  reporting  a  given  amount  of  stocks  and  bonds  than 
they  are  about  reporting  a  similar  amount  in  savings  accounts,  but  if 
stocks  and  bonds  are  predominantly  distributed  among  uncooperative 
higher-income  and  wealthier  persons,  the  completeness  and  accuracy  with 
which  they  are  reported  may  appear  to  be  lower  than  is  the  case  for 
savings  accounts. 

Generalizing  from  interviewer  effectiveness  with  a  set  of  assignments 
to  general  effectiveness  also  necessitates  consideration  of  the  following 
three  problems.  The  first  is  individual  differences.  The  set  of  assign- 
ments received  by  each  interviewer  is,  in  effect,  a  sample  taken  from  the 
population  studied.  Since  people  in  one  sample  may  differ  in  their  willing- 
ness to  cooperate  from  those  in  another  sample,  sampling  variation  must 
be  considered,  particularly  when  interviewers  or  interviewer  groups  are 
compared.    Underlying  the  statistics  for  evaluating  sampling  variation 
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is  the  assumption  of  randomization  of  sample  addresses  among  inter- 
viewers, which  was  not  a  feature  of  the  project  studies. 

The  second  is  the  likelihood  of  interaction  between  persons  contacted 
and  interviewers.  For  example,  lower-income  sample  members  may  be 
more  effectively  interviewed  by  lower-income  interviewers  or  by  inter- 
viewers with  specific  personality  characteristics.  The  observed  effective- 
ness of  a  group  of  interviewers  may  then  depend  on  the  method  of 
selecting  interviewers  as  well  as  on  that  of  selecting  sample  addresses. 

The  third  problem  involves  generalizations  from  the  relationships  ob- 
served for  the  interviewers  employed  in  the  project  to  all  interviewers. 
This  type  of  generalization  assumes  the  random  selection  of  these  inter- 
viewers from  some  undefined  population  of  interviewers. 

Interviewer  Variability:   Studies  Ml  and  M2 

Unfortunately,  no  primary  validation  was  available  in  either  of  these 
studies.  As  a  result,  interviewer  variability  could  only  be  investigated  in 
terms  of  response  rates.  On  Wave  1  of  Study  Ml,  the  response  rate  was 
78  percent,  with  interviewers'  response  rates  ranging  from  45  percent  to 
93  percent.  The  response  rates  of  interviewers  on  Wave  1  of  Study  M2 
ranged  from  50  percent  to  94  percent  with  an  average  of  77  percent.20 

Interviewer  Variability:   Study  M3 

The  interviewer  selection  procedure  was  extensive,  employing  letters 
of  recommendation,  applicant  interviews,  evaluation  checklists,  and 
evaluation  of  pretest  interviews.  Out  of  150  applicants,  only  23  were 
employed.   Nevertheless,  considerable  interviewer  differences  did  appear. 

One  difference  which  became  apparent  was  in  the  honesty  of  the 
interviewers.  Three  interviewers  were  detected  through  the  validation 
data  as  having  falsified  some  of  their  interviews.  These  interviewers  were 
excluded  from  the  analysis.  Two  interviewers  who  made  less  than  10 
contacts  were  also  excluded.  These  interviewers  had  been  hired  on  a 
stand-by  basis  to  fill  in  if  interviewing  help  was  needed. 

Analysis  of  the  remaining  18  interviewers  still  indicated  much  vari- 
ability. These  interviewers  made  309  contacts  on  Wave  1,  or  an  average 
of  about  17  each.  The  proportion  of  contacts  in  which  an  interviewer 
ascertained  the  existence  of  the  validated  savings  account,  the  pick-up 
rate  (response  rate  X  completeness  rate),  ranged  from  32  percent  to  64 
percent  with  an  average  of  50  percent. 

20  Excluded  from  the  response  rates  in  Study  M2  were  two  interviewers  who 
falsified  interviews.  Because  of  the  panel  nature  of  the  study  with  repeated  inter- 
views with  the  same  people,  an  interviewer  faking  data  was  likely  to  be  discovered 
on  subsequent  waves. 
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As  indicated  earlier,  an  interviewer  might  fail  to  obtain  mention  of 
an  account  because  of  a  refusal  or  because  of  the  incomplete  reporting  of 
savings  unit  holdings.  Response  rates  of  interviewers  varied  from  46  per- 
cent to  91  percent,  and  the  completeness  rate  ranged  from  50  percent  to 
90  percent. 

To  what  extent  may  field  procedures  have  produced  the  observed 
differences  between  interviewers?  In  11  cases,  a  panel  member  was  re- 
assigned to  a  different  interviewer  before  the  original  interviewer  made 
contact.  Such  reassignments  resulted  when  an  interviewer  was  delinquent 
in  completing  assignments.  The  exclusion  of  such  reassignments  had  no 
effect  upon  the  results  presented. 

A  second  type  of  reassignment  occurred  when  the  person  talked  to  at 
a  sample  address  refused  to  be  interviewed.  In  these  cases,  the  initial 
interviewer  was  credited  with  a  refusal.  The  new  interviewer  was  credited 
with  an  interview  if  one  was  obtained,  but  if  he  was  refused  he  was  not 
credited  with  a  refusal.  Ten  such  reassignments  occurred  and  four  inter- 
views were  obtained  by  the  interviewers.  A  review  of  these  four  cases 
showed  that  they  did  not  significantly  influence  the  measures  of  inter- 
viewer effectiveness. 

A  possible  weakness  of  the  present  analysis  stems  from  the  process  of 
assigning  sample  addresses  to  interviewers.  For  reasons  of  economy  and 
convenience,  there  was  a  tendency  to  assign  interviewers  addresses  ranging 
from  the  center  of  the  area  outward.  As  a  result,  interviewers  tended  to 
have  a  pie  shaped  cross-section  of  the  area  involving  varying  socioeco- 
nomic levels.  However,  an  examination  of  the  location  of  the  assignments 
of  the  more  effective  interviewers  and  the  location  of  the  assignments  of 
the  less  effective  interviewers  did  not  indicate  concentrations  for  either 
group  in  areas  with  known  socioeconomic  characteristics.21 

Interviewers  were  split  into  three  groups  based  on  the  pick-up  rate. 
There  tended  to  be  some  natural  breaks  in  the  rates  which,  in  part, 
determined  the  grouping  of  interviewers. 

Average 
Com-  Number  of        contacts 

Response        pleteness         Pick-up  inter-  per 

Group  rate  rate  rate  viewers         interviewer 

1 78%  80%  62%  4  19.3 

II 72  72  52  6  17.2 

III 68  50  41  8  16.1 

When  over-all  effectiveness  is  broken  into  its  two  components,  effec- 

21  Further  insights  into  the  heterogeneity  of  interviewer  assignments  can  be 
gained  by  comparing  the  distribution  of  the  size  of  the  validated  time  deposits  of 
sample  members  assigned  to  the  interviewers.  Presentation  of  these  results  will  be 
postponed  until  after  the  grouping  of  interviewers  has  been  discussed. 
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Table  1.   Distribution  of  Accounts,  by  Size  and 

by  Interviewer  Group,  Study  M3 

(Percentages) 


Size  of  account 

Interviewer  groupa 

Total 

I 

II 

III 

Under  $1 ,000 

34 
45 
21 

36 
43 
21 

34 
39 
27 

35 

$l,000-$4,999 

$5,000  and  over 

42 
23 

Total 

100 

100 

100 

100 

77 

103 

129 

309 

a  Group  I  was  more  effective  than  Group  II  and  Group  II  in  turn  was  more  effective  than 
Group  III  as  measured  by  the  pick-up  rate. 

tiveness  at  the  door  as  reflected  by  the  response  rate  and  effectiveness  in 
the  interview  as  reflected  by  the  completeness  rate,  we  observe  that  inter- 
viewers in  Group  I  were  more  effective  in  both  respects  than  were  inter- 
viewers in  the  other  groups.  Those  in  Group  II  were  more  effective  than 
Group  III  interviewers  in  both  respects.  However,  the  pattern  as  we 
move  from  group  to  group  suggests  that  the  less  effective  interviewers  may 
have  been  relatively  better  at  getting  past  the  door  than  the  more  effec- 
tive ones. 

Let  us  turn  now  to  the  question  of  the  heterogeneity  of  the  interviewer 
assignments.  To  what  extent  did  the  distribution  of  the  size  of  savings 
accounts  differ  by  various  interviewer  groups?  All  of  the  differences 
among  interviewer  groups  as  presented  in  Table  1  could  have  occurred 
by  chance.  Therefore,  the  assignment  procedure  employed  did  not  create 
significant  differences  in  the  size-of-account  distribution. 

Although  the  differences  in  Table  1  are  not  significant,  there  was 
some  tendency  for  the  less  effective  interviewers  to  receive  more  assign- 

Table  2.  Results  of  Contacts  for  All  Interviewers, 

by  Size  of  Account,  Study  M3 

(Percentages) 


Measure 


All 
accounts 


Under 
$1,000 


$1,000- 
$4,999 


$5,000 
and  over 


Response  rate 

Completeness  rate . 
Pick-up  rate 

Number  of  contacts 


73 
69 
50 


71 
72 
51 


82 
67 
55 


62 
64 
40 


309 


10^ 


130 


72 
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Table  3.  Pick-up  Rate  (Response  Rate  X  Completeness  Rate) 

by  Size  of  Account  and  by  Interviewer  Group,  Study  M3 

(Percentages) 


Size  of  account 

Interviewer  group* 

I 

II 

III 

Under  SI, 000 

65 
(26)" 

63 
(35) 

56 
(16) 

62 

(37) 
57 

(44) 
32 

(22) 

34 

$1 ,000-$4,999 

(44) 
49 

$5,000  and  over 

(51) 
38 

(34) 

All  contacts 

62 

(77) 

52 
(103) 

41 
(129) 

"  Group  I  was  more  effective  than  Group  III  as  measured  by  the  pick-up  rate. 
b  Number  of  contacts  on  which  percentage  is  based  is  shown  in  parentheses. 


merits  with  savings  units  having  validated  savings  accounts  of  $5,000  and 
over  and  fewer  assignments  with  those  members  having  accounts  of 
$1,000  to  $4,999.  As  Table  2  shows,  savings  units  with  large  accounts 
were  the  least  cooperative,  exhibiting  significantly  lower  response  rates 
than  all  other  savings  units.  Though  one  would  expect  larger  savings 
account  holders  to  be  more  cooperative  in  the  interview  on  the  assump- 
tion that  the  uncooperative  ones  had  refused  at  the  door,  they  were  less 
cooperative.  As  a  result,  the  over-all  cooperativeness  of  the  larger  savings 
account  holders  as  measured  by  the  pick-up  rate  was  also  significantly 
below  that  of  other  savings  units. 

Table  3  shows  how  interviewer  groups  differed  in  their  performance 
at  sample  addresses  where  members  had  different  sizes  of  validated  savings 
accounts.  It  brings  out  a  consistent  pattern  in  which  Group  I  interviewers 
were  more  effective  with  holders  of  all  sizes  of  savings  accounts  than 
either  Group  II  or  Group  III  interviewers.  They  were  significantly  more 
effective  than  Group  III  interviewers  with  contacts  having  under  $1,000 
in  the  validated  account  and  for  all  contacts. 

These  findings  suggest  that  the  differences  between  the  interviewer 
groups  cannot  be  explained  in  terms  of  differences  in  the  assignments 
received  by  each  interviewer.  In  the  remainder  of  this  monograph  it  will 
be  assumed  that  interviewer  differences  were  due  to  interviewer  charac- 
teristics and  interviewer  performance  at  the  door  and  in  the  interview. 


Interviewer  Variability:  Study  M4 

Experience  gained  in  Study  M3  was  utilized  in  planning  and  carrying 
out  Study  M4.   However,  extensive  analysis  of  the  urban  interviewers  had 
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Table  4.  Reporting  of  Different  Assets  and  Debts,  Study  M4 

(Percentages) 


Status 

All 

typesa 

Savings 
accounts 

Checking 
accounts 

Long- 
term 
debt 

Inter- 
mediate- 
term  debt 

Short- 
term 
debt 

Interviewed 

Account 

reported 

Account  not 

reported 

Refused 

89 

68 

21 
11 

75 

52 

23 
25 

92 

89 

3 
8 

96 

77 

19 
4 

91 

82 

9 
9 

91 

58 

33 
9 

Total 

100 

100 

100 

100 

100 

100 

Number  of  cases. .  . 

326 

64 

38 

69 

32 

116 

Percentage  of 

cases 

100 

20 

12 

21 

10 

37 

a  Includes  seven  savings  units  where  two  or  more  assets 
in  breakdown. 


were  validated  which  do  not  appear 


not  been  completed  and  thus  did  not  influence  to  a  large  extent  the  selec- 
tion and  training  of  interviewers  for  the  farm  study.  Because  of  the 
likelihood  that  comparisons  will  be  made  between  these  two  studies, 
emphasis  in  the  subsequent  discussion  will  be  placed  on  pointing  out 
major  differences  between  them. 

Farm  Validation 

Three  types  of  debt  institutions  as  well  as  savings  institutions  were  used 
as  sources  of  sample  names  in  the  farm  study.  About  a  third  of  the  names 
were  selected  from  savings  institutions,  some  from  time  deposit  records, 
and  some  from  demand  deposit  records.  Another  third  of  the  farm  sample 
addresses  were  selected  from  an  institution  offering  a  short-term  revolving- 
type  debt,  and  the  final  third  was  selected  from  two  institutions  offering 
longer-term  debt. 

The  type  of  sample  persons  selected  and  their  willingness  to  discuss  the 
asset  or  debt  validated  varied  by  validation  source.  As  shown  in  Table  4, 
the  pick-up  rate  for  persons  selected  from  savings  account  holders  was 
52  percent;  for  those  selected  from  checking  account  records  it  was  89 
percent;  and  for  those  selected  from  debt  validation  sources  it  ranged 
from  58  percent  for  short-term  debt  to  82  percent  for  intermediate-term 
debt. 

If  the  pick-up  rate  is  broken  down  into  its  components,  the  response 
and  completeness  rates,  further  differences  appear.  Farmers  selected  from 
savings  account  records  and  those  selected  from  short-term  debt  records 
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both  had  low  pick-up  rates,  but  the  response  rate  of  the  former  group 
was  75  percent  in  contrast  to  91  percent  for  the  latter  group.  In  view  of 
similarity  in  the  pick-up  rates  of  these  two  groups,  this  meant  that  farmers 
with  short-term  debt  had  a  lower  completeness  rate  than  did  farmers  with 
savings  accounts.  Farmers  selected  from  demand  deposit  records  and  from 
long-  and  intermediate-term  debt  records  displayed  a  third  pattern  of 
behavior;  for  them,  response  rates  and  completeness  rates  were  both  high. 

These  results  indicated  two  basic  differences  in  the  validation  groups 
used.  First,  the  low  response  rate  for  time-deposit  holders  indicated  that 
they  were  different.  An  analysis  shows  that  they  were  largely  older,  higher- 
income  farmers  or  very  young  farmers.  This  low  response  rate  probably 
reflected  the  opinion  expressed  by  some  middle-aged  farmers  when  asked 
whether  they  had  savings  accounts.  After  saying,  "no,"  they  made  com- 
ments such  as  that  made  by  one  farmer,  "It  doesn't  make  sense  to  put 
money  in  a  savings  account  when  I  can  invest  it  here  on  the  farm  and  get 
a  higher  return." 

The  second  difference  was  found  in  the  sensitivity  of  persons  about 
reporting  a  holding.  A  comparison  of  farmers  selected  from  short-term 
debt  records  with  those  selected  from  other  types  of  debt  records  revealed 
similar  response  rates  of  approximately  92  percent,  but  the  short-term 
debt  group  showed  a  completeness  rate  of  63  percent  while  that  for  the 
other  debt  groups  was  85  percent,  a  difference  significant  at  the  2  per- 
cent level.  Sensitivity  about  mentioning  short-term  debt  may  have  arisen 
because  farmers  may  not  like  to  admit  the  dependency  implied  in  a  short- 
term  loan  whose  repayment  hinges  on  a  good  crop  in  the  fall.  In  contrast, 
a  longer-term  loan  does  not  represent  a  threat  to  the  yearly  solvency  of 
the  farmer. 

The  implications  of  the  different  response  and  completeness  rates  for 
the  study  of  interviewer  effectiveness  are  obvious.  An  attempt  will  be 
made  to  take  this  factor  into  account  in  the  following  analysis  by  setting 
up  two  groups  of  assets  and  debts.  The  one,  called  sensitive  assets,  is 
made  up  of  time  deposits  and  short-term  debt.  The  other,  called  less 
sensitive  assets,  is  composed  of  demand  deposits  and  long-  and  interme- 
diate-term debts. 

Comparison  of  Interviewer  Variability 

The  selection  of  interviewers  for  the  farm  study  was  influenced  to 
some  degree  by  preliminary  findings  from  Study  M3.  The  principal  influ- 
ence was  a  change  in  the  perceived  stereotype  of  the  good  interviewer 
from  the  socially  aggressive  type  to  the  more  matter-of-fact,  pleasant  type. 
This  undoubtedly  tended  to  alter  the  type  of  persons  hired  and  to  create 
greater  homogeneity  among  the  farm  interviewers.    Twenty  interviewers 
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were  selected  from  139  applicants.  However,  one  of  these  was  suspected 
on  the  first  wave  of  falsifying  parts  of  interviews  and  was  excluded  from 
the  present  analysis. 

As  stated  earlier,  the  farm  sample  was  randomly  split  and  half  the 
names  were  assigned  a  change  questionnaire  and  the  other  half  a  holdings 
questionnaire.  Consistent  with  the  findings  in  Study  M3,  a  higher  re- 
sponse rate  was  obtained  on  the  change  than  on  the  holding  approach 
(93  percent  versus  86  percent).  This  appears  to  reflect  the  attitude  of 
interviewers  that  the  change  approach  is  less  prying  and  less  difficult 
to  use. 

Response  rates  of  interviewers  ranged  from  75  to  100  percent  with  a 
median  of  93  percent  on  the  farm  study,  well  above  the  69  percent  of  the 
previous  study.  This  difference  undoubtedly  reflected  the  differing  situa- 
tions and  types  of  persons  involved.  Farmers  working  by  themselves  most 
of  the  day  tended  to  welcome  personal  interaction.  They  were  frequently 
found  at  the  barn  or  in  the  fields  where  a  door  could  not  be  closed  in 
the  interviewer's  face.  Thus,  less  difficulty  was  encountered  in  obtaining 
interviews,  and  door-entrance  techniques  were  of  less  importance,  in 
marked  contrast  to  Study  M3. 

In  the  farm  study  limited  variability  in  response  rates  was  observed, 
but  there  was  considerable  variability  in  completeness  rates.  Since  the 
farmer  by  circumstances  or  nature  was  not  prone  to  refuse  to  be  inter- 
viewed, a  question  arises  as  to  whether  his  refusals  may  possibly  have  come 
more  subtly  by  failing  to  report  information  in  the  interview.  The  com- 
pleteness rat**  for  all  interviews  in  the  farm  study  was  76  percent.  This, 
however,  was  a  misleading  figure  for  two  reasons.  First,  it  included  the 
validation  of  less  sensitive  assets  and  debts  for  over  45  percent  of  the 
farmers.  If  these  were  excluded  and  only  sensitive  assets  and  debts  were 
used,  the  completeness  rate  would  drop  to  64  percent,  which  is  below  the 
69  percent  on  Study  M3.  However,  this  would  understate  the  true  rate 
obtained  by  the  farm  interviewers  because  of  the  second  reason,  an  am- 
biguity in  the  change  questionnaires.22 


22  Part  of  this  difference  may  be  due  to  the  questionnaire  format  for  the  change 
approach  on  the  farm  study  (Appendix  D). 

In  the  section  of  the  change  questionnaire  dealing  with  the  debt  holdings  of 
the  farmer,  the  lead-in  question  read,  "Do  you  have  .  .  .  loans  secured  by  mort- 
gages, sales  contracts,  or  other  loans  on  .  .  . :"  This  was  followed  by  a  list  of  assets 
on  which  the  farmer  might  owe  money.  There  was  a  single  box  for  "no"  answers 
for  each  asset.    The  interviewer  could  interpret  such  "no"  answers  in  three  ways: 

1.  The  farmer  does  not  own  the  particular  asset. 

2.  The  farmer  owns  but  has  no  debt  on  the  asset. 

3.  The  farmer  owns  and  has  debt,  but  there  has  been  no  change  in  the 
amount  owed. 

The  interviewer  was  instructed  in  the  training  sessions  to  check  the  "no"  box 
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The  change  approach  for  all  validations  yielded  a  completeness  rate 
of  65  percent,  while  the  rate  for  the  holdings  approach  was  87  percent. 
Comparable  rates  for  Study  M3  were  67  percent  and  69  percent. 

The  pick-up  rate  on  the  farm  study  depended  on  the  base  used.  The 
over-all  rate  was  68  percent,  which  may  have  been  unduly  low  because 
of  the  possible  ambiguity  in  the  change  approach.  The  rate  for  the 
holdings  approach  only  was  75  percent,  but  this  included  validation  of 
less  sensitive  assets  and  debts  for  almost  half  of  the  farmers.  Possibly  the 
rate  which  allowed  the  most  direct  comparison  with  Study  M3  was  the 
pick-up  rate  for  the  holdings  approach  on  sensitive  assets  and  debts.  This 
figure  was  61  percent,  compared  with  43  percent  on  Study  M3. 

The  pick-up  rate  for  the  holdings  approach  on  the  less  sensitive  assets 
was  90  percent,  indicating  something  of  the  magnitude  of  differences  in 
cooperation  for  sensitive  and  less  sensitive  assets  and  debts. 

Limitations  of  Analysis  of  Interviewer  Effectiveness 

Multiple  validation  sources,  useful  for  certain  aspects  of  the  study, 
became  a  handicap  in  evaluating  interviewer  effectiveness.  Since  the  pro- 
portion of  holdings  to  change  assignments  and  of  sensitive  to  less  sensitive 
validation  assignments  varied  from  one  interviewer  to  another,  meaning- 
ful criteria  could  not  be  developed  unless  these  effects  could  be  removed. 
Yet  when  the  average  of  17  contacts  per  interviewer  was  split  four  ways, 
there  was  an  average  of  about  four  contacts  per  cell,  which  was  obviously 
inadequate  for  evaluating  interviewers  in  view  of  respondent  variability. 

The  number  of  cases  could  be  doubled  if  only  one  split  was  considered 
at  a  time.  Pick-up  rates  were  computed  for  the  sensitive  assets  and  debts 
and  for  the  less  sensitive  ones  for  each  interviewer.  Similar  rates  were 
prepared  on  the  basis  of  the  holding-change  splits.  It  was  hoped  that  the 
more  effective  interviewers  would  tend  to  appear  near  the  top  of  each  of 
these  four  lists  and  the  less  effective  ones  near  the  bottom.  Except  for  a 
few  interviewers,  this  expectation  was  not  met.  Coefficients  of  determina- 
tion for  each  combination  of  pick-up  rates  were  as  follows : 

for  situation  2  and  to  write  <:not  applicable"  for  situation  1.  Situation  3  should 
not  have  been  elicited  until  after  the  name  of  the  person  or  institution  to  whom 
the  debt  was  owed  has  been  obtained  and  entered  on  the  questionnaire.  However, 
in  answering  the  preceding  questions,  a  respondent  may  have  thought  in  terms  of 
change  and  anticipated  questions.  Thus,  a  respondent  may  have  said  in  response 
to  the  initial  question,  "Yes,  I  owe  some  money  on  my  tractor,  but  that  hasn't 
changed."  Unless  the  interviewer  understood  what  was  wanted,  he  might  have 
checked  the  "no"  box  and  failed  to  obtain  information  about  the  creditor.  Such  a 
questionnaire  was  coded  as  not  mentioning  the  validated  asset  or  debt. 
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Less 

Sensitive  sensitive  Holdings  Change 

validations  validations  approach  approach 

1.  Sensitive  validations .003              .004  .053 

2.  Less  sensitive  validations ....               .372  .291 

3.  Holdings  approach ....               ....  .004 

4.  Change  approach ....               ....  .... 

On  the  whole,  the  coefficients  of  determination  are  small,  even  among 
pairs  of  variables  where  there  is  overlap,  such  as  Variables  2  and  3,  which 
have  in  common  all  holdings  forms  which  deal  with  less  sensitive  valida- 
tions. Three  interrelated  explanations  may  be  offered  for  these  low  corre- 
lations. First,  distribution  of  sensitive  and  less  sensitive  validation  assign- 
ments varied  widely  among  interviewers,  which  influenced  pick-up  rates 
on  the  holdings  approach.  Similarly,  the  distribution  of  change  and  hold- 
ings approaches  varied  among  interviewers.  Second,  while  the  average 
number  of  contacts  made  by  each  interviewer  was  1 7,  the  numbers  ranged 
from  12  to  22.  An  interviewer  with  12  contacts  might  only  have  had 
3  or  4  contacts  involving  the  holdings  approach.  Finally,  the  rates  for 
the  less  sensitive  validations  did  not  distinguish  among  interviewers.  Thus, 
there  were  8  interviewers  with  100  percent  rates  and  of  the  remaining  11, 
7  had  rates  of  over  80  percent.  With  the  small  number  of  cases,  these 
limited  differences  are  likely  to  be  unreliable  measures  of  interviewer 
effectiveness. 

Without  any  reliable  measure  of  interviewer  effectiveness  in  the  farm 
study  (M4),  a  rigorous  evaluation  of  the  selection,  training,  and  super- 
vision methods  employed  cannot  be  made.  Much  of  the  following  evalu- 
ation of  methods  is  based  on  the  M3  Study.  This  is  regretted  because  it 
is  felt  that  there  was  considerable  improvement  in  the  methods  used  in 
the  farm  study  over  those  employed  in  Study  M3. 

Summary 

Considerable  interviewer  variability  on  pick-up  and  completeness  rates 
is  evident.  This  suggests  that  there  was  room  for  improvement  in  the 
techniques  and  methods  of  selecting,  training,  and  supervising  interviewers 
which  were  utilized  in  this  study,  yet  they  are  representative  of  the 
methods  employed  in  most  surveys  of  this  type.  The  following  chapters 
describe  the  methods  used  and  suggest  which  were  more  effective  in 
selecting  the  better  interviewers.  Obviously,  with  only  one  study  from 
which  to  draw  conclusions,  the  findings  represent  hypotheses  for  future 
testing. 

Very  little  information  is  available  about  the  applicants  rejected.  As 
a  result,  the  effectiveness  of  the  methods  employed  in  selecting  potentially 
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better  interviewers  from  among  the  applicants  cannot  be  determined. 
This  must  be  kept  in  mind  in  evaluating  the  findings  presented. 

The  reader  must  also  keep  in  mind  the  nature  of  the  asset,  savings 
accounts,  which  was  validated  in  the  M3  Study.  Some  indication  of 
people's  sensitivity  to  reporting  this  asset  can  be  gained  from  the  farm 
study.  While  the  pick-up  rate  using  the  same  interviewers  was  89  percent 
for  checking  accounts,  it  was  only  52  percent  for  savings  accounts.  This 
represents  differences  in  similar  people's  attitudes  toward  reporting  these 
two  types  of  information  to  an  interviewer. 


III.  INTERVIEWER  SELECTION 


Interviewer  selection  procedure  varies  greatly.  As  a  rule,  the  degree  of 
effort  expended  on  this  activity  depends  in  part  on  the  type  of  study  and 
in  part  on  the  researcher  himself.  In  a  relatively  simple  study  with 
structured  questionnaires  and  not  many  questions,  less  effort  is  devoted 
to  interviewer  selection.  In  more  complicated  studies,  greater  care  is 
needed  to  select  interviewers  who  are  able  to  handle  the  details  involved. 

More  generally,  past  experience  seems  to  be  the  principal  guide  used 
in  recruiting  interviewers.  Each  organization  develops  its  own  approach 
to  this  task  based  to  a  large  extent  on  its  experiences  rather  than  on  con- 
trolled experiments.  In  relatively  small  organizations  the  selection  pro- 
cedure is  more  informal  and  is  less  likely  to  make  use  of  personality  tests 
and  other  written  examinations.  The  latter  techniques  appear  to  be  used 
extensively  by  the  large  survey  organizations,  although  the  effectiveness 
of  most  of  these  tests  has  not  been  demonstrated.23 

This  chapter  reports  the  methods,  procedures,  and  techniques  em- 
ployed in  selecting  interviewers  for  the  four  project  studies.  In  general, 
it  is  felt  that  these  procedures,  with  a  few  noted  exceptions,  were  very 
useful  in  selecting  effective  interviewers.  However,  such  subjective  evalu- 
ations are  open  to  possible  bias.  To  complement  these  evaluations,  the 
selection  techniques  used  in  Study  M3  were  evaluated,  where  possible,  in 
terms  of  their  relationship  to  a  measure  of  interviewer  effectiveness,  the 
pick-up  rate. 

The  absence  of  a  significant  relationship  between  these  methods,  pro- 
cedures, and  techniques  and  the  pick-up  rate  should  not  be  interpreted  as 
indicating  that  these  methods  were  of  no  use  in  identifying  potentially 
effective  interviewers.   Most  of  these  variables  were  employed  in  deciding 

23  For  a  description  of  these  testing  procedures,  see  Beatrice  F.  Dvorak,  Frances 
G.  Fox,  and  Charles  Meigh,  "Tests  for  Field  Survey  Interviewers,"  Journal  of 
Marketing,  Vol.  16,  No.  3  (January,  1952),  pp.  301-6;  Stanley  Womer  and  Harper 
Boyd,  "The  Use  of  a  Voice  Recorder  in  the  Selection  and  Training  of  Field 
Workers,"  Public  Opinion  Quarterly,  Vol.  15,  No.  2  (Summer,  1951),  pp.  358-63; 
Principal  Data  Collection  Forms  and  Procedures  (Washington:  U.S.  Bureau  of 
the  Census,  1961);  H.  H.  Lamale,  Methodology  of  the  Survey  of  Consumer  Ex- 
penditures in  1950  (Philadelphia:  University  of  Pennsylvania  Press,  1959),  espe- 
cially pages  63-65;  H.  H.  Hyman  et  al.,  Interviewing  in  Social  Research  (Chicago: 
University  of  Chicago  Press,  1954),  especially  pages  362-63. 
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which  applicants  would  be  hired.  As  a  result,  a  variable  may  have  been 
useful  in  successfully  rejecting  undesirable  applicants  but  may  have  been 
of  no  value  in  discriminating  among  those  hired.  Thus,  the  absence  of  a 
relationship  may  only  reflect  that  the  characteristics  of  interviewers  on 
which  the  analysis  is  based  were  homogeneous  with  respect  to  a  variable 
because  that  variable  was  employed  in  selecting  them. 

Interviewer  Selection  Procedures 

The  discussion  in  this  chapter  refers  only  to  interviewer  selection  for 
the  panel  operations  of  the  Consumer  Savings  Project.  The  other  studies, 
which  were  carried  out  by  the  Survey  Research  Center,  utilized  their 
already-established  field  staff. 

Most  of  the  field  work  for  Study  Ml  was  subcontracted  to  two  well- 
known  interviewing  organizations,  and  the  selection  of  interviewers  was 
the  responsibility  of  these  organizations.  In  addition  to  their  usual  selec- 
tion procedures,  both  organizations  placed  stress  on  procurring  inter- 
viewers with  a  financial  background  and  with  some  knowledge  of  financial 
terminology. 

In  the  remaining  three  panel  operations,  the  field  staff  was  recruited 
by  project  personnel  in  each  of  the  areas  being  sampled.  A  review  and 
evaluation  of  the  recruiting  procedures  used  will  be  given  under  the 
following  headings :  interviewer  qualifications,  sources  of  applicants,  eval- 
uation of  applicants,  selection  of  interviewer  candidates,  evaluation  of 
candidates,  and  final  selection.  Each  of  these  aspects  will  be  covered 
separately  in  the  first  part  of  this  chapter.  The  second  part  will  present 
a  quantitative  method  for  selecting  interviewers  based  on  information 
available  before  applicants  are  formally  hired. 

Interviewer  Qualifications 

For  the  first  two  studies  the  qualifications  sought  in  applicants  for 
interviewing  positions  were  based  on  prior  experience  of  the  project  per- 
sonnel. Included  were  good  grooming,  pleasant  appearance,  good  hand- 
writing, age  between  25  and  55  years,  availability  for  at  least  15  hours  a 
week,  availability  of  an  automobile,  and  permanency  in  the  area. 

The  first  three  of  these  were  obviously  necessary  qualifications.  The 
age  range  was  based  on  current  marketing  literature.  "Those  who  are  too 
young  lack  tact  and  the  ability  to  approach  people  properly.  Those  who 
are  too  old  are  usually  not  willing  to  follow  instructions  carefully  and  are 
not  sufficiently  aggressive."24  The  availability  for  at  least  15  hours  a  week 

24  Lyndon   O.   Brown,  Marketing  and  Distribution   Research    (3rd  ed.;   New 
York:    Ronald,  1955),  p.  301. 
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Table  5.  Record  of  Applicants 


Item 

Study  M3 

Study  M4 

Number  of  Applicants 

141 

136 

Given  a  personal  interview 

Offered  training 

Hired  as  interviewers 

109 
50 
23 
15 

101 
39 
20 

Retained  on  all  five  waves 

15 

was  required  so  that  the  time  schedule  of  the  field  operation  could  be 
maintained.  Since  it  would  be  necessary  for  the  interviewers  to  travel 
considerable  distances  to  obtain  interviews,  a  car  was  essential. 

Permanency  in  the  sample  area  and  willingness  to  remain  on  the 
project  staff  for  the  duration  of  the  study  were  considered  particularly 
important  because  the  same  respondents  were  to  be  reinterviewed  four  or 
five  times.  In  addition,  loss  of  an  interviewer  could  have  meant  new 
training  problems  and  could  have  led  to  antagonizing  the  respondent  by 
giving  a  new  interviewer  access  to  confidential  data.  The  success  of 
adhering  to  this  requirement  is  evidenced  in  Table  5.  On  Study  M3,  15 
of  the  23  interviewers  hired,  or  65  percent,  remained  for  all  five  waves. 
The  retention  rate  on  Study  M4  was  75  percent,  or  15  of  the  20  inter- 
viewers hired. 

Experience  gained  in  Study  Ml  indicated  that  a  financial  background 
was  not  essential.  More  important  was  a  college  education  plus  the  ability 
to  grasp  many  concepts  and  new  ideas;  an  interviewer  with  these  qualifi- 
cations could  be  taught  all  the  financial  terminology  that  was  needed. 

A  final  prerequisite  added  in  Studies  M3  and  M4  was  confidence  that 
financial  data  could  be  obtained  in  the  interview  process.  The  manner 
in  which  this  characteristic  was  evaluated  is  described  in  a  later  section 
of  this  monograph. 

Sources  of  Qualified  Applicants 

School  systems,  state  employment  offices,  and  local  colleges  and  uni- 
versities were  the  primary  sources  of  interviewer  applicants.  Applicants 
were  also  located  through  such  miscellaneous  sources  as  the  Farm  Bureau, 
local  marketing  research  organizations,  recommendations  from  distin- 
guished people  in  the  area,  and  other  previous  studies. 

The  state  employment  offices  were  of  assistance  in  a  number  of  ways. 
They  scanned  their  files  to  see  if  any  of  the  registered  job-seekers  had  the 
necessary  qualifications.  Those  who  did  have  were  asked  to  come  into 
the  employment  office  for  a  personal  interview.  The  offices  also  placed 
ads  in  the  local  newspapers  on  behalf  of  the  study.    In  addition,  the 
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offices  provided  the  project  staff  with  desks  for  personal  interviews  and 
telephones  so  that  incoming  inquiries  could  be  handled  immediately. 
Considerable  time  and  weeding-out  were  required  to  find  qualified  inter- 
viewers through  this  source.  For  example,  the  following  results  were 
obtained  through  one  employment  office:  of  27  responding  to  newspaper 
ads,  17  were  rejected  for  lack  of  basic  qualifications,  10  were  considered 
for  the  job,  but  only  2  were  actually  hired. 

Local  universities  were  used  to  locate  likely  interviewer  applicants  for 
Studies  M2  and  M3.  The  heads  of  the  departments  of  psychology,  soci- 
ology, and  business  administration  were  asked  to  post  announcements  of 
the  positions  and  to  request  professors  and  instructors  to  make  verbal 
announcements  of  the  positions  in  their  graduate  classes.  University  stu- 
dent employment  offices  were  also  contacted,  and  a  number  of  graduate 
students  applied  for  the  interviewing  positions.  A  few  interviewers  were 
located  in  this  way. 

Locating  applicants  through  the  offices  of  superintendents  of  schools 
in  the  districts  and  counties  where  the  studies  were  being  made  was  one 
of  the  most  productive  methods.  In  all  instances,  the  county  superin- 
tendents went  out  of  their  way  to  find  qualified  applicants.  As  one  super- 
intendent remarked,  "The  experience  of  being  with  adults  instead  of  with 
students  would  be  valuable  experience  for  the  teachers  and  would  also 
give  them  a  chance  to  earn  some  needed  extra  money."  The  school 
superintendents  were  of  assistance  in  numerous  other  ways.  They  pro- 
vided up-to-date  listings  of  teachers'  names  and  addresses  so  that  a  post- 
card describing  the  interviewing  work  could  be  sent  to  each  teacher  asking 
those  who  were  interested  to  write  for  further  information.  In  two  coun- 
ties the  superintendents  of  schools  sent  personal  letters  informing  teachers 
of  the  part-time  work  available.  The  letter  sent  by  one  superintendent  of 
schools  asked  a  subjectively  preselected  group  to  return  a  postcard  to  him 
indicating  if  they  were  interested  in  the  assignment.  Since  this  was  a 
select  group,  time  was  saved  in  not  having  to  talk  to  applicants  who  did 
not  have  the  basic  qualifications.  Of  30  teachers  to  whom  letters  were 
sent,  11  indicated  interest;  2  of  these  11  were  selected  and  became  out- 
standing interviewers.  In  one  county  the  superintendent  included  a  de- 
scription of  the  part-time  job  in  his  monthly  directive  sent  to  all  schools 
in  the  county. 

Teachers  were  also  reached  through  teachers'  associations.  One  state 
teachers'  association  was  extremely  helpful  in  addressing  postcard  notices 
of  the  job  openings.  In  view  of  the  excellent  cooperation  obtained  from 
the  school  systems,  it  is  not  surprising  to  note  that  on  the  final  two  panel 
studies  the  majority  of  the  applicants  were  either  school  teachers  or  were 
connected  with  the  school  system,  e.g.,  principals  or  other  administrators. 
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Thus,  in  Study  M3,  76  of  the  141  applicants  (54  percent)  came  from  the 
school  system,  and  in  Study  M4  the  ratio  was  93  of  136  (68  percent). 
In  addition,  much  less  expense  and  effort  were  required  in  obtaining  these 
candidates  than  were  necessary  when  using  other  methods. 

Various  other  sources  also  provided  interviewer  applicants.  These 
included  newspaper  ads  not  linked  with  the  state  employment  service 
(Studies  M2  and  M3),  lists  of  interviewers  used  on  related  prior  studies 
(Study  M3),  and  for  Study  M4,  county  farm  advisers.  A  further  source 
was  the  applicants  themselves.  Those  who  appeared  to  be  good  prospects 
were  told  that  more  people  with  qualifications  similar  to  theirs  were 
needed  and  they  were  encouraged  to  invite  any  such  other  people  whom 
they  knew  to  get  in  touch  with  the  field  director.  No  records  were  kept 
of  applicants  obtained  from  these  miscellaneous  sources. 

It  is  interesting  to  note  that  the  ratio  of  applicants  to  interviewers 
hired  was  about  6  to  1  (see  Table  5) . 

In  terms  of  retention  for  the  entire  operation,  the  number  of  appli- 
cants exceeded  the  number  of  final  interviewers  by  roughly  9  to  1.  Al- 
though most  of  the  weeding  out  occurred  at  the  initial  employment 
interview  stage,  it  is  also  noteworthy  that  roughly  half  of  the  interviewer 
candidates  did  not  survive  the  training  sessions,  which  also  served  as  part 
of  the  selection  process.  This  will  be  discussed  in  a  later  section. 

Evaluation  of  Applicants 

Four  tools  were  used  in  the  evaluation  process,  each  later  converted  to 
a  weighted  numerical  score.  These  four  tools  were  (1)  the  application 
form,  (2)  the  personal  interview  with  an  evaluation  checklist,  (3)  a 
personality  test,  and  (4)  references.  Each  of  these  tools  was  in  turn 
evaluated  from  a  methodological  standpoint,  using  correlation  measures 
between  it  and  a  measure  of  performance  to  determine  to  what  extent 
the  applicant's  score  measured  by  these  tools  was  related  to  subsequent 
performance  measured  by  his  pick-up  rate  as  an  interviewer. 

Application  Form 

The  application  form  served  two  purposes.  It  supplied  facts  about  the 
applicant's  age,  address,  marital  status,  experience,  and  so  on  and  indi- 
cated something  about  the  applicant's  ability  to  follow  simple  instructions 
and  to  do  a  neat  job  in  completing  the  application  form. 

The  application  form  was  an  important  tool  of  the  selection  procedure 
and  merits  some  description.  This  discussion  is  built  around  the  applica- 
tion form  used  on  Study  M4,  shown  in  Figure  1,  although  most  of  the 
questions  appeared  on  application  forms  used  on  earlier  studies. 

Name  in  full  (please  print).    The  complete  name  should  have  been 
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printed  in  full  with  no  nicknames  or  abbreviations  unless  they  were  part 
of  the  applicant's  legal  name. 

Address.  It  was  advisable  to  know  where  the  applicant  lived  in  rela- 
tion to  the  need  for  the  interviewers  and  whether  he  was  centrally  enough 
located  so  as  to  keep  travel  time  to  a  minimum.  The  full  address  was 
needed,  including  an  apartment  number,  if  he  had  one,  and  his  zone 
number. 

How  long?  If  the  applicant  had  lived  at  his  present  address  less  than 
two  years,  some  explanation  was  desirable.  He  may  not  have  been  perma- 
nent in  the  area,  or  he  may  not  have  been  a  stable  individual.  This  was 
one  measure  of  the  applicant's  permanency  which  was  discussed  pre- 
viously. However,  no  relationship  between  this  and  the  interview  pick-up 
rate  was  found  for  Study  M3. 

Home  phone.  The  exchange  and  number  should  have  been  clear  and 
legible,  inasmuch  as  the  applicant  may  have  had  to  be  reached  quickly  to 
clear  up  problems  if  he  became  an  interviewer. 

Office  phone.  If  the  answer  was  "none,"  this  should  have  been  stated. 

Marital  status.  In  all  studies  of  the  project,  married  applicants  were 
given  preference  over  single  ones  if  other  qualifications  were  met.  In  line 
with  common  belief,  it  was  felt  that  married  people  would  be  relatively 
more  stable  and  more  mature  and  thus  make  better  interviewers.  How- 
ever, the  response  rates  and  the  pick-up  rates  did  not  substantiate  this 
belief,  for  both  groups  did  equally  well.  The  coefficient  of  determination 
did  not  differ  significantly  from  zero  on  Study  M3. 

Children  and  ages.  If  there  were  any  children,  there  may  have  been  a 
demand  for  the  applicant  at  home,  thus  reducing  possible  interviewing 
time.  If  the  applicant  was  a  woman  with  very  young  children,  there  was 
little  likelihood  of  hiring  her  unless  she  had  a  baby-sitter  or  unless  some 
other  child-care  arrangements  could  be  made.  Even  if  arrangements 
could  be  made  in  such  a  case,  it  was  felt  that  demands  at  home  might  be 
overwhelming. 

Age.  For  Study  M3  no  significant  coefficient  of  determination  was 
found  between  age  and  pick-up  rate.  Because  of  the  basic  qualifications 
requirement,  the  majority  of  interviewers  selected  were  between  the  ages 
of  25  and  55  —  in  Study  Ml,  93  percent;  in  Study  M2,  96  percent;  in 
Study  M3,  87  percent;  and  in  Study  M4,  90  percent. 

State  of  health.  Questionable  health  had  to  be  explained  because  it 
was  important  for  an  interviewer  to  stay  with  the  panel  study  until  com- 
pleted. If  the  applicant  had  obvious  handicaps  or  disfigurements,  these 
were  noted  because  they  could  have  caused  bias  in  the  information  ob- 
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Figure  1.    Interviewer  Application  Form:    Study  M4 

University  of  Illinois 
Bureau  of  Economic  and  Business  Research 

INTERVIEWER  APPLICATION  FORM 

(Please  print;  answer  all  questions) 

Name  in  full 

Address How  long? 

City Home  phone Office  phone 

□  Single   D  Married   Q  Widowed   D  Divorced 

Number  of  children Ages Your  age State  of  health 

Education:  Check  highest  level  attended. 

r—i  Grade       r—i  High       i— i  Business   r— i  Junior   r-i  College  or  r-Graduate 
school        school      u  school    ^  college    university    work 

If  attended  college,  university  or  business  school: 

Dates  _,       .„ 

attended  Degrees,  if  any 

Name  of  Institution   From  To   Major  field   Minor  field   Name   Date  rec'd 


Full-time  job,  if  any:                          D  None 
Type  of  work , 


Where  employed How  long?_ 

Part-time  jobs,  if  any:  D  None 

Type  of  work 


Where  employed How  long?_ 

Working  days Working  hours:  From T< 

Have  you  lived  on  a  farm?  Q  No  D  Yes    Dates Where 


Have  you  worked  on  a  farm?   D  No   □  Yes     Dates Type  of  work_ 


Have  you  held  a  job  involving 

contact  with  farmers?       U  No   D  Yes     Dates Type  of  work 
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Experience  as  an  interviewer  or  in  related  work  in  past  five  years:    D  None 

Name  of  immediate 
Employer Location      Type  of  work        supervisor 


Number  of  hours  per  week  that  you  could  devote  to  interviewing^ 


Are  you  engaged  in  any  activity  (personal  or  business)  which  takes  you  out  of 
the  area  occasionally?   D  No   Q  Yes    (Explain) 


Any  special  time  during  the  day  or  evening  when  you  would  consistently  not  he 
available  ? 

Any  special  time  of  the  week  when  you  would  consistently  not  be  available? 

Any  special  time  of  the  year  when  you  would  not  be  available? 

Can  you  drive?         Do  you  have  a  car? Is  it  available  at  all  times? 

Do  you  know  shorthand  now? If  yes,  approximate  words  per  minute? 

Can  you  speak  any  foreign  languages  well?  If  yes,  which  ones? 

Why  are  you  applying  for  this  job? 


References:  (Names  and  addresses  of  three  or  more  persons,  not  relatives  or 
personal  friends,  from  whom  information  concerning  your  quali- 
fications or  interviewing  experience  may  be  obtained) 

Circumstances 
Name  Address of  relationship 


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

(Leave  blank) 
Reference  results 


Comments 


Final  evaluation 


34  Survey  Reliability  and  Interviewer  Competence 

tained.  One  of  two  answers  was  usually  given  to  this  state-of-health 
question,  "excellent"  or  "good."  The  more  effective  interviewers  were 
not  concentrated  in  either  group. 

Education.  Applicants  with  college  degrees  were  preferred.  There 
was  no  requirement  as  to  the  type  of  degree,  although  it  was  preferable 
that  the  degree  be  in  a  field  somewhat  related  to  this  work:  finance, 
economics,  marketing,  accounting,  sociology,  psychology,  and  so  on.  Two 
other  factors  to  be  noted  were  the  time  elapsed  since  the  applicant 
received  the  degree  and  whether  the  college  education  had  been  uninter- 
rupted. If  it  had  been  interrupted,  there  should  have  been  some  expla- 
nation. Following  are  the  average  response  rates  for  Study  M2  inter- 
viewers compared  with  the  level  of  education  completed. 

High  school  or  less  (7  interviewers) 70.3% 

College  (6  interviewers) 85.2 

Postgraduate  work  (3  interviewers) 88.3 

In  the  light  of  these  findings,  applicants  with  at  least  a  college  education 
were  sought  for  the  two  subsequent  studies.  Of  the  Study  M3  inter- 
viewers, 96  percent  had  at  least  a  college  education  and  of  the  Study  M4 
interviewers,  100  percent  were  college  graduates.  However,  no  relation- 
ship was  discovered  between  the  pick-up  rates  and  education  of  Study  M3 
interviewers. 

Full-time  job,  if  any.  It  was  expected  that  the  applicant  would  al- 
ready have  a  full-time  job  unless  the  applicant  was  a  housewife.  It  was 
helpful  if  the  full-time  job  was  in  a  related  field,  such  as  finance  or  ac- 
counting, or  in  a  position  where  there  was  some  contact  with  the  general 
public  such  as  sales,  social  work,  or  teaching. 

Where  employed?  Where  the  applicant  worked  in  relationship  to 
where  he  lived  might  have  caused  a  travel  problem  with  interviewing 
work. 

How  long?  This  was  one  measure  of  the  applicant's  stability.  If  he 
had  been  employed  less  than  six  months,  the  previous  employment  was  to 
be  specified,  together  with  the  reason  for  leaving  the  previous  job. 

Part-time  jobs.  All  part-time  jobs  were  to  be  listed  since  they  could 
have  interfered  with  the  applicant's  ability  to  work  as  an  interviewer. 
In  Study  M3,  no  significant  coefficient  of  determination  between  this 
variable  and  the  pick-up  rate  was  obtained. 

Farm  background.  Three  questions  related  to  farming  were  used  only 
for  the  farm  study.  Since  the  interviewer  was  to  be  talking  to  farm  owners 
and  operators,  some  knowledge  of  farming  was  essential.  If  the  applicant 
had  no  farm  background,  he  was  not  qualified  as  an  interviewer  for  this 
particular  study. 


Interviewer  Selection  35 

Experience.  There  was  overlapping  on  this  question  if  the  applicant 
had  previously  recorded  experience  under  full-time  or  part-time  jobs.  It 
was  important  that  the  name  of  the  immediate  supervisor  be  listed  so  the 
supervisor  could  be  used  as  a  reference  if  desired.  No  relationship  ap- 
peared to  exist  between  previous  interviewing  or  related  experience  and 
the  interviewer's  pick-up  rate  for  Study  M3. 

Number  of  hours.  The  expected  number  of  hours  per  week  was  15  to 
20  even  though  the  interviewer  was  more  likely  to  work  10  to  15  hours. 
After  Study  M3  was  completed,  the  interviewers  were  queried  on  the 
average  number  of  hours  a  week  they  actually  worked  when  interviewing. 
They  reported  between  12  and  14  hours  a  week.  Since  the  interviewing 
was  completed,  the  interviewers  had  nothing  to  lose  by  being  honest. 

In  direct  contrast  to  these  reports,  on  Wave  1  of  Study  M2  the  inter- 
viewers worked  an  average  of  1 7V4  hours  a  week.  This  was  based  on  total 
time  actually  worked  and  reported  by  the  interviewers  on  their  time  sheets. 
Since  these  interviewers  were  paid  by  the  hour,25  the  results  were  not 
very  surprising.  In  addition,  the  unstructured  nature  of  this  study  re- 
quired that  the  interviewers  type  up  complete  details  of  the  interview, 
thus  raising  the  average  hours  worked  per  week. 

If,  on  the  application  form,  less  than  15  hours  was  reported  as  avail- 
able for  interviewing,  the  applicant  was  usually  rejected.  A  report  of 
20  or  more  hours  a  week  was  considered  unrealistic,  especially  if  the  appli- 
cant had  a  full-time  job  and  a  family. 

Some  applicants  recorded  "no  limit"  or  "any  amount  of  time."  This 
was  not  adequate,  and  they  were  pressed  for  the  number  of  hours  they 
could  devote  to  work  outside  of  their  regular  activities.  When  pressured, 
a  few  such  applicants  said  "five  hours"  or  "five  or  ten  hours,"  which  was 
clearly  not  enough.  Some  applicants  did  not  answer  the  question  directly, 
but  tried  to  hedge  until  they  found  out  the  number  of  hours  they  were 
expected  to  work.  The  number  of  hours  reported  on  the  application  blank 
did  not  correlate  significantly  with  the  pick-up  rate  on  Study  M3.  Since 
the  purpose  of  this  question  was  obvious,  an  effort  was  made  to  estimate 
how  much  time  the  interviewer  would  have,  depending  on  his  existing 
obligations.  For  each  interviewer  in  Study  M3  an  over-all  estimate  of 
weekly  hours  available  was  calculated.  This  estimate  yielded  a  significant 
coefficient  of  determination  (.135)  with  interviewer  response  rate,  al- 
though it  was  not  significantly  related  to  pick-up  rate. 

Engaged  in  other  activities.  Other  activities,  either  personal  or  busi- 
ness, which  would  take  the  applicant  out  of  the  area  occasionally  could 

25  See  the  section  beginning  on  page  81  for  the  results  of  paying  interviewers 
by  the  hour  compared  with  paying  them  by  the  completed  interview. 
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have  been  detrimental  to  the  completion  of  field  assignments.  If  the 
answer  was  "yes"  to  this  question,  a  complete  explanation  was  required. 

Special  times  not  available.  Three  questions  were  asked  relating  to 
any  time  during  the  day  or  evening,  week,  or  year  when  the  applicant 
would  not  be  available.  If  the  answer  was  "yes"  to  any  of  these,  a  full 
explanation  was  required.  If  the  applicant  said  "no"  to  all  three  ques- 
tions, he  was  asked  about  special  obligations  (such  as  clubs,  organizations, 
meetings,  and  vacations).  "No"  was  an  easy  (and  the  expected)  answer 
to  this  question,  so  it  was  probed  carefully.  Even  if  an  applicant  said 
"yes"  to  one  or  more  of  these  questions  it  did  not  mean  that  he  should 
not  be  selected,  although  these  obligations  might  have  limited  his  inter- 
viewing activities. 

Car.  As  was  mentioned  previously,  a  car  was  necessary  for  this  type 
of  interviewing,  and  it  had  to  be  available  at  all  times  so  the  interviewer 
could  make  and  keep  appointments.  If  the  applicant  was  a  woman,  it  was 
better  if  the  car  was  her  own  and  not  the  family  car,  which  might  have 
been  needed  at  the  same  time  she  had  an  interviewing  appointment. 

Shorthand.  Ability  to  take  shorthand  was  clearly  desirable  for  an 
interviewer.  However,  other  qualities  seemed  much  more  important  and 
this  ability  was  not  a  prerequisite  for  hiring  and  was  not  taken  into  con- 
sideration unless  two  interviewers  were  equal  in  all  other  respects. 

Foreign  language.  The  purpose  of  a  question  on  foreign  language  was 
to  have  this  information  on  file  in  case  sample  members  were  encountered 
who  only  spoke  a  foreign  language. 

Reasons  for  applying.  This  question  was  asked  only  on  the  application 
forms  for  Studies  M3  and  M4.  Applicants  were  separated  into  three 
groups  on  the  basis  of  their  answers:  financial  reasons,  social  or  interest 
reasons,  or  a  combination  of  financial  and  social  reasons.  Response  and 
pick-up  rates  for  both  studies  were  not  significantly  related  to  these  three 
groups  of  reasons. 

References.  The  instructions  concerning  references  specified  clearly 
"three  or  more  persons,  not  relatives  or  personal  friends."  Yet  many 
applicants  listed  only  one  or  two  references  or  listed  relatives  or  personal 
friends.  All  references  were  reviewed  with  the  applicant  and  the  legibility 
of  names  and  addresses  was  checked.26 

The  space  at  the  bottom  of  the  page  was  used  for  reference  results, 
comments,  and  final  evaluation  of  the  applicant. 

The  application  was  generally  completed  at  the  time  of  the  personal 
interview.    The  completed  form  was  evaluated  and  classified  into  one  of 

20  See  the  section  starting  on  page  44  for  further  details  on  the  use  of  refer- 
ences in  the  selection  of  interviewers. 
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the  following  categories:  (1)  good  prospect;  (2)  possibly  hire;  (3)  pos- 
sibly reject;  or  (4)  reject  this  prospect. 

The  factors  given  greatest  consideration  in  this  subjective  decision 
were  neatness,  ability  to  follow  instructions,  location,  length  of  time  at 
present  address,  marital  status,  age,  number  and  ages  of  children,  state  of 
health,  education,  type  of  work  at  present  job,  length  of  time  at  present 
job,  interviewing  experience,  weekly  hours  available,  other  activities,  time 
not  available,  car,  and  reasons  for  applying. 

The  value  of  using  an  application  form  was  unquestionable.  However, 
no  way  of  compiling  answers  or  of  weighting  them  was  discovered  which 
led  to  a  score  which  was  significantly  related  to  interviewer  pick-up  rates 
on  Study  M3. 

Personal  Interview 

The  purpose  of  the  personal  interview  was  to  gather  information 
about  the  applicant  to  aid  in  the  selection  of  interviewer  candidates.  For 
the  Consumer  Savings  Project  all  personal  interviews  were  conducted  by 
the  project  director,  the  field  director,  and  a  university  graduate  assistant. 
An  effort  was  made  to  standardize  the  personal  interviewing  procedures. 
The  interviews  were  held  in  semi-private  facilities  of  the  state  employment 
offices,  chambers  of  commerce,  and  hotels. 

The  personal  interview  was  comprised  of  four  parts  and  required  15 
to  45  minutes,  generally  averaging  about  25  minutes.  Included  were  a 
review  of  the  application  form,  a  brief  description  of  the  job  with  an 
effort  to  evaluate  the  interviewer's  confidence  that  he  could  do  the  work, 
a  personality  test,  and  an  evaluation  checklist.  Each  of  these  will  be 
covered  in  order. 

The  application  form  was  reviewed,  with  the  interviewer  probing  for 
further  details  in  areas  of  interest  or  wherever  the  application  form  had 
not  been  filled  in  adequately.  The  brief  job  description  covered  such 
subjects  as  the  purpose  and  objectives  of  the  study,  the  information  being 
sought,  the  time  and  interviewing  schedule,  the  training  procedures,  the 
approximate  length  of  the  interviews,  the  number  of  assignments,  the  need 
for  confidential  treatment  of  the  data,  and  the  compensation.  At  the  same 
time,  in  Studies  M3  and  M4  an  attempt  was  made  to  ascertain  the  reac- 
tion of  the  applicants  to  this  type  of  work  and  whether  obtaining  personal 
financial  data  was  something  which  they  felt  could  be  done.  They  were 
asked,  "Do  you  think  people  will  give  this  information?"  "If  not,  why 
not?"  Although  no  attempt  was  made  to  evaluate  interviewers  on  the 
basis  of  their  answers,  pessimistic  or  negative  attitudes  of  applicants  were 
noted. 

Each  applicant  was  asked  to  take  a  personality  test.    A  discussion  of 
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Figure  2.    Evaluation  Checklist:    Study  M4 

University  of  T1.li.nois 
Bureau  of  Economic  and  Business  Research 

Applicant  Evaluation 
Personal  Interview  Check  List 


Characteristic 

Superior 

Above 
average 

Average 

Below 
average 

Appearance  (dress) 

Voice 

Poise 

Self-confidence 

Intelligent  questions 

Intelligent  answers 

Likable  (friendly) 

Attitude  towards  this 
study- 

Ob  o'ectivity 

Physical  appearance 

Applicant 


Interviewer 


Date 
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the  evaluation  of  the  results  of  these  personality  tests  will  be  covered  in 
the  section  beginning  on  "page  49.  While  the  applicant  was  taking  this 
personality  test,  the  project  representative  was  filling  out  an  evaluation 
checklist,  shown  in  Figure  2.  The  purpose  of  this  last  part  of  the  inter- 
view was  to  call  attention  to  any  personal  characteristics  that  might  have 
been  overlooked.  In  addition,  if  the  project  representative  felt  that  he 
did  not  have  enough  information  about  the  applicant  to  rate  him  on  one 
or  more  of  these  characteristics,  further  questions  could  be  asked  after 
the  completion  of  the  personality  test. 

Immediately  after  the  applicant  had  left,  the  project  representative 
reviewed  the  evaluation  form  and  assigned  it  one  of  the  following  four 
ratings:  (1)  hire  this  applicant;  (2)  possibly  hire;  (3)  possibly  reject;  or 
(4)  reject  this  applicant. 

The  pick-up  rates  for  Study  M3  were  used  to  evaluate  this  tool  in  the 
selection  process;  however,  no  significant  relationship  was  discovered.  In 
addition  to  the  over-all  checklist  evaluation,  each  characteristic  was  corre- 
lated with  the  pick-up  rates  for  Study  M3.  A  significant  coefficient  of 
determination  was  not  obtained  for  any  of  these  characteristics. 

Personality  Tests 

For  Studies  M2  and  M3,  interviewers  were  given  the  Inventory  of  Per- 
sonal Attitudes  (IPA)  test,  which  consisted  of  30  statements  for  which 
the  interviewer  was  to  indicate  whether  he  would  "strongly  agree," 
"agree,"  "disagree,"  or  "strongly  disagree."  The  test  was  to  cover  the 
following  attitudes  toward  life:  future  time  orientation,  desirability  of 
stopping  to  think,  originality,  opinion  that  life  (world)  is  predictable, 
belief  in  fate,  and  willingness  to  try  many  actions.  This  test  did  not  prove 
useful  for  identifying  good  interviewers. 

After  Study  M2  interviewers  had  been  hired,  they  were  given  the 
Edwards  Personal  Preference  Schedule  (EPPS)  test,  which  consisted  of 
225  sets  of  two  statements.  The  interviewer  was  to  select  the  statement 
which  he  liked  the  most  or  disliked  the  least.  The  EPPS  was  developed 
to  provide  measures  of  15  relatively  independent  personality  variables. 
A  complete  description  of  each  of  these  variables  is  presented  in  Figure  3. 

The  results  of  this  test  were  used  to  aid  in  the  selection  of  interviewers 
for  Study  M4.  Each  applicant,  on  the  basis  of  his  scores  on  dominance 
(high  score  desirable)  and  abasement  (low  score  desirable)  variables,  was 
put  into  one  of  the  following  four  groups:  (1)  hire  this  applicant, 
(2)  possibly  hire;  (3)  possibly  reject;  or  (4)  reject  this  applicant.  How- 
ever, this  particular  semi-subjective  approach  did  not  prove  an  optimal 
indicator  for  identifying  interviewers  with  a  high  pick-up  rate. 

One  question  of  interest  was  whether  the  interviewer  selection  pro- 
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Figure  3.  Definitions  of  Variables  Available  from  the 
Edwards  Personal  Preference  Schedule 


/.  ach  Achievement:  To  do  one's 
best,  to  be  successful,  to  accomplish 
tasks  requiring  skill  and  effort,  to  be 
a  recognized  authority,  to  accomplish 
something  of  great  significance,  to  do 
a  difficult  job  well,  to  solve  difficult 
problems  and  puzzles,  to  be  able  to 
do  things  better  than  others,  to  write 
a  great  novel  or  play. 

2.  def  Deference:  To  get  sugges- 
tions from  others,  to  find  out  what 
others  think,  to  follow  instructions 
and  do  what  is  expected,  to  praise 
others,  to  tell  others  that  they  have 
done  a  good  job,  to  accept  the  leader- 
ship of  others,  to  read  about  great 
men,  to  conform  to  custom  and  avoid 
the  unconventional,  to  let  others 
make  decisions. 

3.  ord  Order:  To  have  written 
work  neat  and  organized,  to  make 
plans  before  starting  on  a  difficult 
task,  to  have  things  organized,  to 
keep  things  neat  and  orderly,  to 
make  advance  plans  when  taking  a 
trip,  to  organize  details  of  work,  to 
keep  letters  and  files  according  to 
some  system,  to  have  meals  organized 
and  a  definite  time  for  eating,  to 
have  things  arranged  so  that  they  run 
smoothly  without  change. 

4.  exh  Exhibition:  To  say  witty 
and  clever  things,  to  tell  amusing 
jokes  and  stories,  to  talk  about  per- 
sonal adventures  and  experiences,  to 
have  others  notice  and  comment 
upon  one's  appearance,  to  say  things 
just  to  see  what  effect  it  will  have  on 
others,  to  talk  about  personal  achieve- 
ments, to  be  the  center  of  attention, 
to  use  words  that  others  do  not  know 
the  meaning  of,  to  ask  questions 
others  cannot  answer. 

5.  aut  Autonomy:  To  be  able  to 
come  and  go  as  desired,  to  say  what 
one  thinks  about  things,  to  be  inde- 
pendent  of   others   in   making   deci- 


sions, to  feel  free  to  do  what  one 
wants,  to  do  things  that  are  uncon- 
ventional, to  avoid  situations  where 
one  is  expected  to  conform,  to  do 
things  without  regard  to  what  others 
may  think,  to  criticize  those  in  posi- 
tions of  authority,  to  avoid  responsi- 
bilities and  obligations. 

6.  aff  Affiliation:  To  be  loyal  to 
friends,  to  participate  in  friendly 
groups,  to  do  things  for  friends,  to 
form  new  friendships,  to  make  as 
many  friends  as  possible,  to  share 
things  with  friends,  to  do  things  with 
friends  rather  than  alone,  to  form 
strong  attachments,  to  write  letters  to 
friends. 

7.  int  Intraception:  To  analyze 
one's  motives  and  feelings,  to  observe 
others,  to  understand  how  others  feel 
about  problems,  to  put  one's  self  in 
another's  place,  to  judge  people  by 
why  they  do  things  rather  than  by 
what  they  do,  to  analyze  the  behavior 
of  others,  to  analyze  the  motives  of 
others,  to  predict  how  others  will  act. 

8.  sue  Succorance:  To  have  others 
provide  help  when  in  trouble,  to  seek 
encouragement  from  others,  to  have 
others  be  kindly,  to  have  others  be 
sympathetic  and  understanding  about 
personal  problems,  to  receive  a  great 
deal  of  affection  from  others,  to  have 
others  do  favors  cheerfully,  to  be 
helped  by  others  when  depressed,  to 
have  others  feel  sorry  when  one  is 
sick,  to  have  a  fuss  made  over  one 
when  hurt. 

9.  dom  Dominance:  To  argue  for 
one's  point  of  view,  to  be  a  leader  in 
groups  to  which  one  belongs,  to  be 
regarded  by  others  as  a  leader,  to  be 
elected  or  appointed  chairman  of 
committees,  to  make  group  decisions, 
to  settle  arguments  and  disputes  be- 
tween others,  to  persuade  and  influ- 
ence others  to  do  what  one  wants,  to 
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supervise  and  direct  the  actions  of 
others,  to  tell  others  how  to  do  their 
jobs. 

10.  aba  Abasement:  To  feel  guilty 
when  one  does  something  wrong,  to 
accept  blame  when  things  do  not  go 
right,  to  feel  that  personal  pain  and 
misery  suffered  does  more  good  than 
harm,  to  feel  the  need  for  punish- 
ment for  wrong  doing,  to  feel  better 
when  giving  in  and  avoiding  a  fight 
than  when  having  one's  own  way,  to 
feel  the  need  for  confession  of  errors, 
to  feel  depressed  by  inability  to  han- 
dle situations,  to  feel  timid  in  the 
presence  of  superiors,  to  feel  inferior 
to  others  in  most  respects. 

11.  nur  Nurturancc:  To  help 
friends  when  they  are  in  trouble,  to 
assist  others  less  fortunate,  to  treat 
others  with  kindness  and  sympathy, 
to  forgive  others,  to  do  small  favors 
for  others,  to  be  generous  with  others, 
to  sympathize  with  others  who  are 
hurt  or  sick,  to  show  a  great  deal  of 
affection  toward  others,  to  have  oth- 
ers confide  in  one  about  personal 
problems. 

12.  chg  Change:  To  do  new  and 
different  things,  to  travel,  to  meet 
new  people,  to  experience  novelty 
and  change  in  daily  routine,  to  exper- 
iment and  try  new  things,  to  eat  in 
new  and  different  places,  to  try  new 
and  different  jobs,  to  move  about  the 
country  and  live  in  different  places, 


to     participate     in     new     fads     and 
fashions. 

13.  end  Endurance:  To  keep  at  a 
job  until  it  is  finished,  to  complete 
any  job  undertaken,  to  work  hard 
at  a  task,  to  keep  at  a  puzzle  or  prob- 
lem until  it  is  solved,  to  work  at  a 
single  job  before  taking  on  others,  to 
stay  up  late  working  in  order  to  get 
a  job  done,  to  put  in  long  hours  of 
work  without  distraction,  to  stick  at 
a  problem  even  though  it  may  seem 
as  if  no  progress  is  being  made,  to 
avoid  being  interrupted  while  at 
work. 

14.  het  Heterosexuality:  To  go  out 
with  members  of  the  opposite  sex,  to 
engage  in  social  activities  with  the 
opposite  sex,  to  be  in  love  with  some- 
one of  the  opposite  sex,  to  kiss  those 
of  the  opposite  sex,  to  be  regarded  as 
physically  attractive  by  those  of  the 
opposite  sex,  to  participate  in  discus- 
sions about  sex,  to  read  books  and 
plays  involving  sex,  to  listen  to  or  to 
tell  jokes  involving  sex,  to  become 
sexually  excited. 

15.  agg  Aggression:  To  attack  con- 
trary points  of  view,  to  tell  others 
what  one  thinks  about  them,  to  criti- 
cize others  publicly,  to  make  fun  of 
others,  to  tell  others  off  when  disa- 
greeing with  them,  to  get  revenge  for 
insults,  to  become  angry,  to  blame 
others  when  things  go  wrong,  to  read 
newspaper  accounts  of  violence. 


cedure  obtained  people  having  EPPS  scores  different  from  the  adult  pop- 
ulation or  from  the  college  population  used  to  set  norms  for  the  test.  A 
comparison  of  Columns  1,  2,  and  5  in  Table  6  indicates  that  it  did.  Inter- 
viewers selected  on  Studies  M327  and  M4  showed  significantly  more 
"intraception"  and  "dominance"  than  the  adult  population.  The  manifest 
needs  associated  with  intraception  as  shown  in  Figure  3  are  "to  analyze 
one's  motives  and  feelings,"  "to  observe  others,  and  "to  put  one's  self  in 
another's  place."    Needs  associated  with  dominance  are,  "to  argue  for 

27  Two  interviewers  used  on  Wave  1  of  Study  M3  were  no  longer  working 
when  the  EPPS  was  administered.  As  a  result,  the  averages  for  Study  M3  were 
based  on  16  instead  of  18  interviewers. 
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Table  6.   Norms  for  Selected  Populations  and 

Average  Interviewer  Scores  on  the  Edwards 

Personal  Preference  Schedule 


EPPS 
variables 

Adult 
sample 

(1) 

College 
sample 

(2) 

Study 
M3 

(3) 

Study 
M4 

(4) 

Studies 

M3  and 

M4 

(5) 

Study 
M4 

rejects 
(6) 

14.1 

14.5 

15.2 

12.1 

13. 0a 

16.3 

14. 8" 

11.9° 

12. 2C 

15.9° 

17. 2a 

15.1 

16.7 

10.5 

11.5 

11.5 

14.4 

11.8 

10.2 

14.3 

13. 3a 

16.2 

16.7° 

11.6° 

15.8 

13.7 

15.2 

16.4 

12.6° 

16. 0C 

11.7 

11.6 

15.5 
14.8 
13.2 
12.6 
12.9 
15.0 
18.8 
7.9 
15.5 
12.2 
14.5 
17.8 
14.6 
14.4 
10.4 
11.4 

14.5 
14.6 
14.3 
12.7 

9.7b 
16.5 
19.0 

9.2 
17.2 
11.9 
15.9 
15. 2b 
19. 2b 
10. 2b 

9.8 
11.0 

15.0 
14.7 
13.8 
12.7 
11.2 
15.8 
18.9 
8.6 
16.3 
12.0 
15.2 
16.5 
17.1 
11.3 
10.1 
11.2 

15.2 

Deference 

Order 

14.8 
13.4 
14.4 

Autonomy 

10.3 
16.7 

Intraception 

18.3 
9.7 

Dominance 

Abasement 

15.7 
13.3 
14.8 

Change 

16.2 

Endurance 

Heterosexuality 

Aggression 

Consistency 

16. 7d 

9.2 

11.6 

11.5 

8,963 

1,509 

16 

19 

35 

37 

a  Significant  difference  at  the  5  percent  level  between  Study  M4  and  Study  M3  interviewers 
and  the  college  and/or  adult  population  as  indicated. 

b  Significant  difference  at  the  5  percent  level  between  Study  M4  and  Study  M3  interviewers. 

c  Significant  difference  at  the  1  percent  level  between  Study  M4  and  Study  M3  interviewers 
and  the  college  and/or  adult  population  as  indicated. 

d  The  lower  endurance  scores  of  the  Study  M4  rejects  as  compared  with  those  accepted  is 
significant  at  the  5  percent  level. 


one's  point  of  view,"  "to  be  a  leader  in  groups  to  which  one  belongs,"  and 
"to  be  regarded  by  others  as  a  leader." 

The  interviewers  used  on  the  two  studies  showed  less  of  the  following 
than  the  general  adult  population:  autonomy,  succorance,  abasement, 
and  nurturance.  Autonomy  is  associated  with  a  need  "to  be  independent 
of  others  in  making  decisions,"  "to  feel  free  to  do  what  one  wants,"  "to 
criticize  those  in  positions  of  authority,"  or  "to  avoid  responsibilities  and 
obligations."  Succorance  is  associated  with  a  need  "to  seek  encouragement 
from  others,"  "to  have  others  provide  help  when  in  trouble,"  or  "to  have 
others  feel  sorry  when  one  is  sick."  Needs  associated  with  abasement  are, 
"to  feel  guilty  when  one  does  something  wrong,"  "to  accept  blame  when 
things  do  not  go  right,"  and  "to  feel  that  personal  pain  and  misery  suf- 
fered does  more  good  than  harm."  Clearly,  excessive  amounts  of  these 
characteristics  are  not  desirable  in  interviewers.  Nurturance  is  associated 
with  the  following  needs:  "to  help  friends  when  they  are  in  trouble," 
"to  treat  others  with  kindness  and  sympathy,"  and  "to  show  a  great  deal 
of  affection  toward  others."   While  at  first  it  may  be  thought  that  this  is  a 
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desirable  characteristic  for  interviewers,  there  is  a  possibility  that  persons 
with  this  characteristic  are  too  expressive  and  have  strong  needs  for 
nurturance  themselves. 

A  comparison  of  the  selected  interviewers'  scores  with  those  of  the 
college  population  showed  that  the  former  had  significantly  more  intra- 
ception  and  endurance.  Endurance  is  described  as  the  need  "to  keep  at  a 
job  until  it  is  finished,"  "to  work  hard  at  a  task,"  or  "to  stick  at  a  problem 
even  though  it  seems  as  if  no  progress  is  being  made."  The  selected  inter- 
viewers showed  less  of  the  following  than  the  college  population:  succor- 
ance  and  heterosexuality.  The  latter  is  associated  with  needs  "to  go  out 
with  members  of  the  opposite  sex,"  and  "to  kiss  those  of  the  opposite  sex." 
This  attribute  probably  reflects  the  stage  of  the  life  cycle  and,  hence,  social 
maturity. 

Thus,  the  selection  process  appeared  to  have  obtained  persons  with 
desirable  interviewer  traits.  If  so,  what  part  of  the  selection  procedure 
was  responsible?  Some  insight  into  this  question  can  be  gained  by  com- 
paring the  applicants  hired  on  the  farm  study  with  those  rejected.  A 
review  of  Columns  4  and  6  in  Table  6  indicates  that  with  the  exception 
of  endurance,  the  applicants  rejected  did  not  differ  significantly  from  the 
applicants  hired. 

It  appears  then  that  the  selection  of  interviewers  who  differed  from 
the  general  adult  population  in  characteristics  measured  by  the  EPPS 
primarily  occurred  not  because  of  letters  of  recommendation  or  personal 
interview  evaluation  or  other  aspects  of  the  formal  selection  procedure, 
but  because  of  the  nature  of  the  newspaper  ad  and  the  ad  sent  to  school 
teachers,  and  because  a  certain  type  of  person  tends  to  apply  for  inter- 
viewing jobs.28  These  findings  should  not  be  interpreted  to  mean  that  the 
letters  of  recommendation,  evaluations  of  the  applicants  in  personal  inter- 
views, and  test  scores  were  not  useful  in  selecting  effective  interviewers  but 
rather  that  they  were  generally  ineffective  in  further  selection  based  on 
the  characteristics  measured  by  the  EPPS. 

A  comparison  of  the  characteristics  of  the  Study  M4  interviewers  with 
those  of  the  Study  M3  interviewers  shows  that  the  former  had  significantly 
higher  scores  on  the  endurance  test  and  lower  scores  on  the  autonomy, 
change,  and  heterosexuality  tests.  Of  these,  only  change  has  not  been 
described.  It  involves  the  need  "to  do  new  and  different  things,"  "to 
travel,"  "to  meet  new  people,"  "to  experience  novelty  and  change  in  daily 
routine,"  "to  experiment  and  try  new  things,"  and  "to  participate  in  new 
fads  and  fashions."   On  the  surface,  it  would  seem  that  these  are  desirable 

28  This  is  not  completely  true  since  a  few  applicants  were  not  given  the  EPPS 
because  they  were  too  young  or  too  old  or  because  they  were  obviously  unfit  for 
the  job. 
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characteristics  for  interviewers.  They  are  continually  meeting  new  people 
and  experiencing  new  situations.  However,  change  has  a  second  dimen- 
sion, and  this  one  relates  to  instability,  immaturity,  and  the  like.  The 
absence  of  the  latter  characteristics  is  possibly  of  greater  importance  in  an 
interviewer  than  the  presence  of  the  former  ones.  The  characteristics  of 
the  Study  M4  interviewers  when  contrasted  with  those  of  the  Study  M3 
interviewers  were  not  inconsistent  with  the  change  in  interviewer  stereo- 
type from  the  more  emotional,  socially  aggressive  type  of  interviewer  to 
the  friendly,  matter-of-fact  type  which  occurred  as  a  result  of  the  analysis 
of  the  interviewers  used  in  Study  M3. 

Of  the  15  attributes  measured  by  the  tests  comprising  the  EPPS,  only 
one,  dominance,  was  significantly  related  to  the  pick-up  rates  of  inter- 
viewers. A  coefficient  of  determination  of  .36  was  obtained.  In  other 
words,  the  dominance  scores  of  the  16  interviewers  explained  36  percent 
of  the  variance  in  their  pick-up  rates.  Subsequent  analysis,  reported  later, 
indicated  an  interaction  between  dominance  scores  and  abasement  scores. 

References 

All  references  other  than  personal  friends  and  relatives  were  sent  a 
mimeographed  one-page  letter  requesting  "a  frank  statement  on  the  en- 
closed sheet  concerning  this  person's  qualifications  for  this  work  with  par- 
ticular reference  to  the  characteristics  listed."  The  sheet  enclosed,  shown 
in  Figure  4,  sought  ratings  of  the  applicant  with  regard  to  a  number  of 
different  characteristics  that  were  thought  to  be  related  to  interviewer 
ability.  In  addition,  a  blank  space  at  the  top  of  the  sheet  provided  room 
for  the  reference  to  write  comments  in  his  own  words  about  the  appli- 
cant's fitness  for  the  job. 

The  applicant  evaluations  were  scored  by  assigning  numerical  values 
to  the  ratings  for  each  characteristic,  ranging  from  four  for  "superior" 
down  to  one  for  "below  average."  These  scores  were  averaged  and  each 
applicant  was  then  assigned  to  one  of  the  same  four  groups  mentioned  in 
connection  with  the  evaluation  which  resulted  from  the  personal  inter- 
view. However,  no  relationship  was  observed  between  this  measure  and 
the  pick-up  rate. 

Coefficients  of  determination  were  prepared  for  all  the  characteristics 
on  the  evaluation  forms  in  relation  to  the  pick-up  rates  for  interviewers 
used  on  Study  M3.  Of  these,  the  references'  evaluations  of  self-confidence 
and  of  appearance,  manner,  and  poise  were  significantly  related  to  pick-up 
rates.  The  coefficient  of  determination  for  self-confidence  was  .34,  indi- 
cating that  34  percent  of  the  variance  in  the  interviewers'  pick-up  rates 
was  explained  by  this  variable.  The  coefficient  of  determination  for  ap- 
pearance was  .29.    This  suggests  that  further  refinements  in  a  reference 
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Figure  4.  Reference  Evaluation  Form  :   Study  M4 


University  of  Illinois 
Bureau  of  Economic  and  Business  Research 


APPLICANT  EVALUATION 


Personal  evaluation  of 


A.  Information  about  the  applicant  which  you  believe  would  be  valuable  in  evaluating 
this  individual  for  work  as  an  interviewer: 


B.  Following  is  a  list  of  characteristics  which  all 
For  each  of  these,  please  indicate  how  you  would 


individuals  have  in  varying  degrees, 
rate  this  applicant. 


Characteristic 

Superior 

Above 
average 

Average 

Below 
average 

Inadequate 

basis  for 

judgment 

Ability  to  make  friends 

Honesty 

Self-confidence 

Attention  to  detail 

Tendency  to  procrastinate 

Ability  to  think  on  feet 

Appearance,  manner,  and  poise 

Initiative 

Signed 


Date 
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evaluation  form  might  yield  high  returns  in  terms  of  selecting  or  identi- 
fying potentially  effective  interviewers. 

Selection  of  Interviewer  Candidates 

The  selection  of  interviewer  candidates  was  based  on  the  following 
scoring  system  (using  each  of  the  four  tools  employed  in  the  evaluation  of 
applicants)  with  the  following  weights: 

Method  of  Possibly  Possibly 

evaluation  Hire  hire  reject  Reject 

Application  form 8  6  4                 2 

Checklist  evaluation 16  12  8                 4 

Personality  test  (M4  only) 8  6  4                 2 

References 16  12  8                 4 

As  is  evident  from  this  weighting  system,  at  the  time  the  selection  was 
made  more  reliance  was  placed  on  the  checklist  evaluation  and  on  the 
information  obtained  from  references  than  on  the  personality  test  or  on 
the  application  form.  The  actual  procedure  consisted  of  ranking  the 
applicants  by  their  total  scores  and  selecting  as  many  as  were  needed 
beginning  from  the  top  of  the  list.  At  the  same  time,  certain  additional 
factors  were  taken  into  consideration,  such  as  the  location  of  the  appli- 
cant relative  to  the  concentration  of  the  sample  members. 

Whether  this  selection  procedure  did  in  fact  select  the  most  effective 
interviewers  could  not  be  determined.  Since  no  measure  of  performance 
was  available  for  those  applicants  who  were  rejected,  it  could  not  be 
shown  that  the  best  interviewers  were  selected. 

That  this  selection  procedure  was  effective  could  only  be  measured  in 
the  response  and  completeness  rates  obtained  on  the  later  studies  where 
the  selection  process  was  much  improved  and  used  effectively.  What  was 
apparent  from  the  interviewers  selected  was  that  much  interviewer  varia- 
bility remained,  and  thus  there  was  much  room  for  improvement  in  the 
selection  procedure. 

Once  selected,  applicants  were  sent  a  letter  offering  them  training29 
(with  pay)  and  the  possibility  of  being  hired  if  they  performed  satisfac- 
torily in  the  training  sessions.  A  letter  was  also  sent  to  applicants  who 
were  rejected,  leaving  open  the  possibility  that  they  might  be  hired  at  a 
future  time. 

Evaluation  of  Candidates  Based  on  Training  Experience 

Candidates  were  evaluated  on  the  basis  of  their  conduct  at  the  train- 
ing sessions,  on  their  scores  on  a  qualifying  examination,  and  on  their 

29  Refer   to   Table   5    for   a  breakdown   of   the   numbers   offered    training   on 
Studies  M3  and  M4. 
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performance  in  trial  interviews.  Interviewer  candidates  were  expected  to 
attend  all  training  sessions  and  were  penalized  for  lack  of  attendance 
unless  unusual  circumstances  were  involved.  In  addition,  each  candidate's 
general  conduct  was  observed  and  evaluated.  Did  the  candidate  dress 
appropriately,  take  written  notes,  listen  attentively,  ask  questions,  get 
along  with  the  other  candidates,  and  prepare  for  the  meetings?  No  at- 
tempt was  made  to  score  each  of  these  points.  However,  a  candidate's 
performance  in  these  respects  was  used  as  a  guide  in  deciding  whether  or 
not  to  hire  him  as  an  interviewer.  The  specific  content  of  these  training 
sessions  will  be  discussed  in  the  following  chapter. 

After  the  first  three  training  sessions,  candidates  were  given  a  take- 
home  "Interviewer  Qualifying  Examination,"  shown  in  Appendix  A.30 
This  examination  was  to  be  completed  and  returned  to  the  project  office 
within  one  week.  The  examination  was  designed  in  such  a  way  that  the 
majority  of  the  questions  could  be  answered  directly  from  the  written 
training  materials.  However,  some  of  the  questions  related  to  situations 
described  verbally  in  the  meetings,  and  other  questions  introduced  prob- 
lem situations  similar,  though  not  identical,  to  those  covered  in  the 
meetings. 

Essentially,  the  qualifying  examination  served  two  purposes.  On  the 
one  hand,  it  brought  out  how  well  the  candidate  had  familiarized  himself 
with  the  training  materials  and  had  absorbed  the  discussions  at  the  train- 
ing sessions,  relating  not  only  to  interviewing  techniques  but  also  to  ac- 
quaintanceship with  financial  terminology.  On  the  other  hand,  the 
examination  forced  the  interviewer  to  consult  the  written  materials  (if  he 
expected  to  pass)  and  thus  become  better  acquainted  with  this  material, 
thereby  serving  as  an  additional  training  device. 

The  qualifying  examination  was  scored  before  the  fourth  training 
session  and  was  returned  and  discussed  at  that  session.  The  scores  served 
as  a  basis  for  weeding  out  poor  candidates  as  well  as  a  basis  for  highlight- 
ing areas  in  which  other  candidates  were  weak.  The  examination  also 
caused  some  candidates  to  drop  out  of  the  study  because  they  felt  it  was 
too  hard.  Although  the  main  purpose  of  the  examination  was  its  use  as  a 
training  device,  it  was  also  used  in  the  selection  process.  For  those  inter- 
viewers who  were  selected  for  Study  M3,  correlation  was  not  significant 
between  their  test  scores  and  their  interviewing  ability  as  measured  by 
their  pick-up  rates.  This,  of  course,  should  not  discredit  the  value  of  the 
qualifying  examination  since  its  main  purpose  as  a  teaching  tool  was 
satisfied. 

Trial  interviews  served  as  an  additional  means  of  candidate  evalua- 


30  This  examination  was  used  in  Study  M4.    Essentially  the  same  examination 
had  been  used  in  Studies  M2  and  M3. 
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tion.  Three  to  five  such  interviews  were  assigned  to  each  candidate  at  the 
end  of  the  third  training  session.  The  completed  interviews  were  to  be 
returned  to  the  project  office  within  ten  days.  These  interviews  were 
conducted  in  the  same  manner  as  regular  interviews,  with  the  same  ad- 
vance letter.  After  being  checked  and  edited,  they  were  returned  to  the 
candidate.  They,  too,  served  as  a  basis  for  discussion  in  the  fourth  train- 
ing session. 

In  Study  M2,  some  of  the  trial  interviews  were  dummy  interviews  in 
the  sense  that  arrangements  were  made  beforehand  with  people  in  the 
sample  area  to  act  as  respondents.  These  people  were  not  coached  on 
what  answers  to  give.  They  were  told  to  act  natural  and  report  only  what 
the  interviewer  requested,  if  they  were  so  inclined.  Following  the  inter- 
view, these  respondents  informed  the  field  director  about  the  information 
reported  to  the  interviewer.  In  addition,  they  reported  on  the  attitude  of 
the  interviewer.  An  evaluation  was  then  made  of  the  interviewer's  ability. 
Although  this  experiment  was  adopted  only  in  the  pilot  study,  it  proved 
to  be  of  assistance  in  selecting  interviewers  as  well  as  in  providing  insights 
into  the  interview  situation  from  the  point  of  view  of  the  respondent. 
Numerous  problems  involved  in  using  the  device  limited  its  value.  The 
first  problem  which  really  had  two  aspects,  was  that  of  locating  respond- 
ents to  play  the  role  and  briefing  them.  Second,  and  probably  more  im- 
portant, was  the  problem  of  developing  an  adequate  standard  of  evalua- 
tion for  the  dummy  interviews.  Two  other  minor  problems  were  those  of 
time  and  cost.  However,  taking  these  problems  into  consideration,  it  was 
felt  that  the  dummy  interviews  were  a  valuable  tool  in  selecting  effective 
interviewers. 

In  Studies  M3  and  M4  no  dummy  interviews  were  attempted  because 
time  and  other  restrictions  prevented  the  making  of  necessary  arrange- 
ments. 

Final  Selection  of  Interviewers 

Candidates  were  selected  as  interviewers  only  if  they  performed 
satisfactorily  on  all  three  aspects  of  the  evaluation  process.  In  Study  M3, 
23  interviewers  were  selected  from  50  candidates;  in  Study  M4,  20  inter- 
viewers were  selected  from  39  candidates  (Table  5).  The  majority  of 
the  candidates  who  were  not  selected  eliminated  themselves  by  quitting 
because  of  the  pressures  of  training  sessions,  qualifying  examinations,  and 
trial  interviews. 

Once  an  interviewer  was  selected,  he  was  asked  to  read  and  sign  an 
"Interviewer  Agreement,"  which  outlined  the  nature  of  the  interviewing 
task,  the  duties,  and  the  arrangements  for  compensation  (see  Appendix 
B).  At  the  same  time,  it  was  made  clear  verbally  that  continuation  on 
the  project  was  contingent  on  satisfactory  interviewing  performance. 
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Table  7.    Distribution  of  Project  Interviewers 

by  Selected  Characteristics  and  by  Study 

(Percentages) 


Characteristic  category 

Study 
Ml 

Study 

M2 

Study 
M3 

Study 
M4 

Sex 

Male 

29 

58 

70 

70 

Female 

71 

42 

30 

30 

Age 

Under  25 

7 
71 

4 
73 

9 

57 

5 

25-40 

60 

41-55 

22 

23 

30 

30 

56  and  over 

0 

0 

4 

5 

Education 

High  school  or  less 

43 

27 

4 

0 

College 

50 

62 

44 

80 

Postgraduate 

7 

11 

52 

20 

Occupation 

Teacher 

0 

11 

39 

45 

Other  professional 

36 

31 

22 

5 

Housewife 

43 

31 

9 

20 

Other 

21 

27 

30 

30 

Base  (number  of  interviewers) 

14 

26 

23 

20 

Table  7  shows  the  distribution  of  interviewers  hired  for  the  various 
studies  by  selected  characteristics.  As  is  evident  from  this  table,  the  great 
majority  were  men  (except  in  Study  Ml),  between  the  ages  of  25  and  40 
years  of  age,  college  educated  (an  appreciable  number  had  postgraduate 
work),  and  employed  in  professional  occupations.  The  heavy  ratio  in 
favor  of  men  was  surprising  because  no  attempt  had  been  made  to  favor 
one  sex  over  the  other. 


A  Quantitative  Method  for  Selecting  Interviewers 

To  what  extent  can  the  information  obtained  about  applicants  and 
trainees  be  utilized  to  select  the  potentially  more  effective  interviewers? 
The  method  employed  for  predicting  interviewer  effectiveness  was  linear 
regression  using  an  equation  of  the  following  form: 

y  =  a  +  b-tZ-L  +  b2z2  +  •  •  •  +  bkzb  +  u, 

where  y  was  the  measure  of  interviewer  effectiveness,  the  z's  were  appli- 
cant data  variables,  and  u  was  a  random  variable.  The  6's  were  estimated 
from  the  n  observations  of  interviewer  characteristics  on  zt  and  y.  If  the 
us  were  normally  distributed,  the  least  squares  method  also  yielded  a 
maximum  likelihood  estimate. 
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The  analysis  employed  the  data  from  Study  M3.  Although  it  would 
have  been  desirable  to  test  the  regression  equations  obtained  from  the 
Study  M4  data,  this  was  not  feasible  in  view  of  the  difficulty  in  identifying 
the  effective  interviewers  on  that  study,  as  was  previously  discussed.  In 
addition,  the  interviewers  were  a  fairly  homogeneous  group.  Applicants 
with  adverse  letters  of  recommendation  or  with  adverse  evaluation  were 
not  hired.  Applicants  under  25  or  over  50  and  those  with  less  than  a 
college  education  were  rejected  unless  they  had  other  compensating 
characteristics.  In  general,  persons  employed  in  social  types  of  work,  such 
as  school  teachers,  were  given  preference.  Those  doing  very  poorly  on 
the  take-home  qualifying  examination  were  not  hired.  For  these  reasons 
and  because  of  the  small  number  of  interviewers  employed  in  Study  M3, 
the  equations  presented  are  little  more  than  hypotheses  for  future  testing. 

The  following  variables,  discussed  earlier,  were  evaluated  in  an  at- 
tempt to  find  combinations  which  might  prove  useful  in  the  selection  of 
interviewers. 

1.  Sex. 

2.  Age. 

3.  Education. 

4.  State  of  health  (applicant's  evaluation). 

5.  Length  of  residence  at  present  address. 

6.  Length  of  time  at  present  job. 

7.  Over- all  estimate  of  number  of  hours  available  for  interviewing 
each  week. 

8.  Financial  background  of  interviewer. 

9.  Previous  interviewing  experience. 

10.  Reasons  for  applying  for  an  interviewing  position. 

1 1 .  Neatness  of  the  application  form  and  extent  to  which  the  applicant 
followed  directions  in  completing  the  form. 

12.  Each  of  the  10  scales  making  up  the  evaluation  checklist  form. 

13.  Each  of  the  8  scales  making  up  the  reference  rating. 

14.  The  EPPS  scores  on  each  of  the  15  tests. 

An  IBM  1401  was  employed  to  search  for  the  best  regression  model. 
Even  so,  many  combinations  of  variables  could  not  be  evaluated.  The 
variables  which  in  the  previous  section  showed  a  relationship  with  the 
pick-up  rate  were  reference  evaluation  of  "self-confidence,"  the  "domi- 
nance" score  on  the  EPPS,  and  the  checklist  evaluation  of  "likableness." 
Information  on  some  of  these  variables  was  not  obtained  for  all  inter- 
viewers. The  first  measure  was  available  for  all  18  interviewers,  the 
second  one  for  16  interviewers,  and  the  last  one  for  11  interviewers. 
Since  most  data  were  available  for  the   16  interviewers  who  took  the 
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EPPS,  these  were  used  as  the  basis  for  the  following  regression  equations. 

Several  criteria  were  employed  in  selecting  the  regression  equations 
presented.  One  was  the  percentage  of  variance  in  the  pick-up  rates  of 
interviewers  explained  by  the  regression  equation.  This  was  the  coefficient 
of  multiple  determination.  Another  was  the  ratio  of  net  regression  coeffi- 
cients to  their  standard  errors.  This  ratio  combined  with  an  evaluation 
of  beta  coefficients  gives  some  indication  of  the  reliability  and  importance 
of  specific  variables.  Finally,  the  reliability  of  the  estimate  was  evaluated 
using  the  standard  error  of  the  estimate. 

It  was  found  that  the  two  variables,  self-confidence  (references)  and 
dominance  (EPPS),  had  to  be  present  before  the  equations  proved  satis- 
factory on  the  basis  of  the  criteria  applied.  When  these  two  variables 
were  combined  with  the  over-all  estimate  of  the  number  of  hours  available 
for  interviewing,  the  following  multiple  regression  equation  was  obtained. 

Pick-up  rate  =  2.89  +  4.217  zx  +  .853  z2  +  .224  z3 

(1.727)       (.281)       (.091) 
R2  =    .71 
Su  =  4.58 

In  this  equation  zt  was  the  reference  evaluation  of  self-confidence  of  the 
applicant,  z2  was  the  score  on  the  EPPS  dominance  test,  z3  was  the  over- 
all estimate  of  hours  available  for  interviewing,  and  2.89  was  the  constant 
term.  The  number  in  parenthesis  below  each  net  regression  coefficient 
is  the  standard  error  of  the  coefficient.  The  R2  is  the  corrected  coefficient 
of  multiple  determination,  and  Su  is  the  corrected  standard  error  of  the 
estimate.31 

The  respective  beta  coefficients  were  Bx  =  .422,  B2  =  .538,  and  B3  = 
.400. 

These  variables  explained  about  71  percent  of  the  variance  in  the 
interviewers'  pick-up  rates.  This  differed  significantly  from  zero,  as  did 
each  of  the  net  regression  coefficients. 

If  the  interviewers  were  divided  into  thirds  on  the  basis  of  their  pre- 
dicted pick-up  rate,  the  actual  pick-up  rate  for  each  group  would  reflect 
the  effectiveness  of  the  equation  in  differentiating  among  interviewers. 

Predicted                                       Completeness  Pick-up 

pick-up  rate                                          rate  rate 

Highest  third 79%  58% 

Middle  third 66  49 

Lower  third 58  41 

All  interviewers 68  50 

31  Mordecai  Ezekiel  and  Karl  A.  Fox,  Methods  of  Correlation  and  Regression 
Analysis  (New  York:    Wiley,  1959),  pp.  300-2.  tmiuFRSfll  Of  ^uN°*5 
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Thus,  the  pick-up  rate  for  the  highest  third  would  be  1 7  percent  above 
that  for  the  lowest  third.  In  addition,  the  middle  third  would  be  about 
midway  between  the  two  extreme  groups,  suggesting  a  continuous  differ- 
entiation among  interviewers. 

These  findings  suggest  that  the  selection  process  may  be  made  more 
rigorous.  However,  the  findings  presented  require  further  validation  on 
larger  groups  of  interviewers. 


IV.  INTERVIEWER  TRAINING 


The  amount  and  intensity  of  interviewer  training  required  tends  to 
vary  with,  among  other  things,  the  complexity  of  the  study,  the  famili- 
arity of  the  interviewers  with  the  subject  matter,  the  type  of  question- 
naire being  used,  and  the  availability  of  time  and  other  resources  for 
adequate  instruction.  On  highly  specialized  studies,  such  training  may  be 
extensive,  especially  in  the  case  of  a  panel  operation.  Training  may  be 
given  by  mail,  by  phone,  by  personal  contact  with  individuals,  or  in  group 
meetings.  As  a  rule,  group  meetings  are  preferable,  if  feasible,  since  it  is 
only  in  this  way  that  standardization  of  procedures  may  be  achieved. 

The  training  procedures  on  the  later  Consumer  Savings  Project  studies 
evolved  out  of  the  experiences  gained  on  the  earlier  studies.  Although 
it  is  felt  that  these  training  methods  were  good,  the  dimensions  of  a 
training  program  are  hard  to  quantify  and  evaluate  objectively. 

Various  goals  may  be  sought  in  training  sessions.  The  following 
warrant  particular  attention,  and  were  also  those  to  receive  the  most 
emphasis  in  the  training  sessions  for  the  panel  studies  of  the  Consumer 
Savings  Project. 

1.  To  appreciate  the  value  of  economic  research. 

2.  To  develop  an  understanding  and  appreciation  of  the  particular 
study. 

3.  To  develop  the  ability  to  obtain  interviews,  with  special  emphasis 
on  the  possible  approaches  to  reluctant  sample  members  and  to  those 
unwilling  to  cooperate. 

4.  To  instill  an  understanding  of  the  need  for  accurate  and  complete 
data  and  of  the  techniques  and  methods  by  which  such  information  can 
be  obtained. 

5.  To  familiarize  the  interviewers  fully  with  the  questionnaire,  with 
the  purpose  and  nature  of  the  particular  questions,  and  with  dimensions 
within  which  answers  are  to  be  sought. 

6.  To  build  morale  —  to  convey  to  the  interviewer  the  feeling  that  he 
is  a  member  of  a  group. 

7.  To  weed  out  unqualified  interviewers  by  observing  the  ability  of 
each  candidate  to  absorb  training  materials  and  to  conduct  satisfactory 
interviews. 
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8.  To  promote  standardization  of  procedures  and  of  the  type  of  an- 
swers to  be  sought  for  the  different  questions. 

A  variety  of  techniques  are  available  for  achieving  these  objectives, 
ranging  from  the  assignment  of  background  reading  material  to  the  con- 
ducting of  trial  interviews  and  assigning  interviewers  to  edit  and  code 
questionnaires.32  A  number  of  these  procedures  were  used  in  the  training 
sessions  of  the  Consumer  Savings  Project. 

Interviewer  Training  for  Consumer  Financial  Studies 

Interviewers  employed  on  the  Consumer  Savings  Project  had  to  be 
thoroughly  trained  for  two  major  reasons.  First,  the  study  was  concerned 
with  savings,  assets,  and  debts,  which  are  subjects  about  which  many 
respondents  are  extremely  sensitive.  Second,  because  of  the  experimental 
nature  of  the  project,  its  success  was  highly  dependent  on  good  field 
work.  The  interviewers  had  to  understand  the  purpose  of  the  study  and 
had  to  follow  instructions.  If  they  were  confronted  with  new  situations 
not  covered  in  the  training,  they  had  to  be  able  to  make  the  correct 
decision  on  the  basis  of  their  training  and  understanding  of  the  goals  of 
the  project. 

The  training  sessions  for  each  pilot  study  incorporated  the  experience 
gained  from  the  training  sessions  for  the  preceding  pilot  studies. 

Instructions  for  most  types  of  situations  were  covered  in  the  training 
manual.  They  were  also  reviewed  verbally  and  thus  reinforced.  At  the 
beginning  of  each  study,  the  interviewers  were  given  initial  training 
which  consisted  of  four  meetings  2'/2  to  3  hours  in  length.  A  training 
session  was  also  held  before  each  new  wave  of  interviews,  followed  on 
some  of  the  waves  of  later  studies  by  a  "review  session"  after  the  inter- 
viewers had  had  a  chance  to  complete  a  few  interviews.  In  addition, 
meetings  were  held  at  the  completion  of  each  wave  to  review  experi- 
ences on  that  wave. 

Initial  Training  Sessions 

The  initial  meetings  had  two  purposes:  to  train  each  interviewer  in 
the  techniques  and  methods  needed  to  accomplish  the  objectives  of  the 

32  For  a  discussion  of  some  of  these  approaches  see  Robert  L.  Kahn  and 
Charles  F.  Cannell,  The  Dynamics  of  Interviewing  (New  York:  Wiley,  1957), 
pp.  241-51;  Lester  Guest,  "A  New  Training  Method  for  Opinion  Interviewers," 
Public  Opinion  Quarterly,  Vol.  18,  No.  3  (Fall,  1954),  pp.  287-99;  U.S.  Bureau 
of  the  Census,  Principal  Data-Collection  Forms  and  Procedures  (Washington: 
U.S.  Government  Printing  Office,  1961);  Muriel  Harris,  compiler,  Documents 
Used  During  the  Selection  and  Training  of  Social  Survey  Interviewers  and  Se- 
lected Papers  on  Interviewers  and  Interviewing  (London:  Central  Office  of  Infor- 
mation, The  Social  Survey,  1956). 
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Figure  5.   Interviewer  Guide:    Study  M4 
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study  and  to  weed  out  those  who  apparently  could  not  learn  to  obtain 
good  interviews. 

In  addition  to  the  general  goals  mentioned  earlier,  these  training 
sessions  had  the  objective  of  instructing  the  interviewers  in  the  meanings 
of  financial  terms.  When  the  respondent  gave  information  about  specific 
financial  holdings  (debentures,  endowment  life  insurance,  and  so  forth), 
the  interviewer  needed  to  know  what  the  respondent  was  talking  about. 
Knowledge  of  the  financial  terms  used  could  serve  to  facilitate  communi- 
cation between  the  interviewer  and  the  respondent  and  make  the  collec- 
tion of  accurate  and  complete  data  more  probable. 

These  objectives  were  achieved  in  a  number  of  ways.  At  the  first 
meeting,  each  interviewer  was  given  a  packet  containing  an  "Interviewer 
Guide,"  a  set  of  "Pointers  on  the  Interview  Situation,"  a  schedule  of 
material  to  be  covered  in  all  the  meetings,  and  the  field  forms  to  be  used 
on  the  first  wave  of  interviews.  Tables  of  contents  of  the  "Guide"  and 
"Pointers"  for  Study  M4  are  shown  in  Figures  5  and  6;  Figure  7  presents 
the  schedule  of  materials  covered  in  the  first  three  training  sessions  for 
that  study. 

The  "Interviewer  Guide"  was  designed  and  indexed  so  that  inter- 
viewers could  use  it  themselves  to  answer  questions  after  formal  training 
was  completed.   Its  substantive  aim  was  threefold. 

1.  To  convey  a  clear  explanation  of  what  the  study  was  all  about  — 
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Figure  6.   Outline  of  "Pointers  on  The  Interview  Situation"  : 
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Refusals  4 

Excuses  6 

Sickness  or  Serious  Illness  8 

Appointments  8 

General  Problems  on  Contact  Results  9 

6.  Noncontact  Situations  and  How  to  Handle  9 

Moved  or  Not  There  9 

Not  Home  9 

B.  The  Interview 

1.  Introduction  and  Explanation  10 

2.  Creating  and  Maintaining  Rapport  1 1 

3.  Type  of  Data  Sought  12 

4.  Refusals  and  "Don't  Know"  Answers  12 

5.  Completing  the  Interview  14 

6.  Use  of  Records  15 


its  aims  and  objectives,  the  types  of  data  needed,  and  how  the  data  fitted 
in  with  the  aims  and  objectives. 

2.  To  present  a  glossary  of  financial  terms. 

3.  To  discuss  the  forms  to  be  used  in  the  first  wave  of  interviews,  first 
in  general  terms  to  indicate  the  purpose  and  functions  of  each,  and 
then  question  by  question  to  point  out  possible  sources  of  misunderstand- 
ing and  to  make  clear  what  information  was  being  sought. 

Judging  from  the  interviewers'  reactions  to  the  "Guide,"  it  served  its 
practical  purpose  of  helping  them  obtain  complete,  accurate  information. 
As  one  interviewer  stated,  "Although  the  initial  set  of  instructions  seemed 
overwhelming,  they  proved  to  be  highly  useful  and  were  of  considerable 
aid." 

The  pamphlet,  "Pointers  on  the  Interview  Situation,"  was  designed  to 
supplement  the  "Interviewer  Guide"  by  focusing  on  the  procedures  to  be 
followed  in  obtaining  an  interview  with  the  sample  member.  In  addition 
to  discussing  means  of  approaching  the  sample  member,   a  substantial 
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portion  of  this  material  (Sections  A4  to  A6  in  Figure  6)  was  devoted  to  a 
listing  of  different  contact  situations  that  might  be  encountered,  with 
specific  suggestions  for  handling  each  situation.  In  Studies  M3  and  M4, 
these  "case  situations"  were  based  on  actual  interview  situations  experi- 
enced in  earlier  studies.  Thus,  practical  examples  were  undoubtedly 
helpful  in  increasing  the  number  of  interviews  obtained. 

The  schedule  outlined  in  Figure  7  indicates  the  subject  matter  covered 
in  the  first  three  training  sessions  and  the  order  of  discussion.  These 
sessions  were  held  on  successive  days  (usually  in  the  evening).  The  first 
session  was  devoted  to  a  review  of  the  study,  the  means  of  securing  the 
cooperation  of  the  sample  member,  the  time  schedule  for  that  wave,  and 
the  schedule  of  interviewer  compensation.  Special  attention  was  given 
to  the  last  of  these  aspects  in  Studies  M3  and  M4,  payment  for  which 
was  made  on  an  interview  basis  geared  to  an  incentive  system,  as  will  be 
explained  in  Chapter  V. 

The  main  focus  of  the  second  session  was  on  the  forms  to  be  used, 
with  emphasis  on  an  item-by-item  review  of  the  questionnaire  and  of  the 
interviewer  report  form.  The  final  part  of  this  session  was  concerned 
with  a  discussion  of  operating  procedures  both  in  contacting  respondents 
and  in  filling  out  and  returning  forms. 

The  third  session  was  devoted  to  a  general  review  of  the  study  objec- 
tives and  questionnaire  forms  and  to  role-playing.  At  the  end  of  that 
session,  candidates  were  handed  copies  of  the  qualifying  take-home 
examination  and  were  given  pretest  interview  assignments. 

The  fourth  and  final  session  was  held  about  ten  days  after  the  first 
three.  In  the  interim,  the  qualifying  examination  and  the  pretest  inter- 
views had  been  completed  and  a  decision  had  been  made  as  to  which 
candidates  were  to  be  hired.  Only  the  latter  were  invited  to  the  fourth 
session.  Even  with  the  care  taken  at  the  previous  three  sessions  to  give 
the  interviewer  a  solid  background,  there  were  many  problems  to  be 
discussed  at  this  session.  The  interviewers  benefited  from  the  opportunity 
to  talk  over  their  problems  and  experiences  after  they  had  been  exposed 
to  some  of  the  actual  field  work. 

The  first  part  of  the  meeting  was  used  to  further  instruct  the  inter- 
viewers in  the  weak  areas  indicated  by  the  qualifying  examination  and 
the  trial  interviews.  The  interviewers  were  encouraged  to  discuss  the 
problems  they  encountered  and  how  they  solved  them.  The  corrected 
copies  of  the  qualifying  examination  were  returned,  and  there  was  a 
review  of  the  most  frequently  missed  questions.  Interviewers  could  also 
ask  about  specific  questions  which  were  marked  wrong  but  not  reviewed. 

Next,  the  meeting  turned  to  reviewing  and  correcting  the  trial  inter- 
views.   Each  interviewer  was  given  one  of  his  completed   assignments 
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Figure  7.   Schedule  Outline  for  Training  Sessions:    Study  M4 

I.  First  Session 

A.  Distribution  of  all  materials 

B.  Introduction  of  field  staff  and  interviewers 

C.  Discussion  of  time  schedule  form 

D.  Discussion  of  interviewer  compensation  form 

E.  Presentation  of  facts  about  the  study  (Interviewer  Guide  —  Part  A) 

1.  Objectives 

2.  Sponsorship 

3.  Reason  for  project 

4.  Selection  of  sample 

5.  Panel  operation 

6.  Experimental  aspects 

F.  Explanation  of  terms  (Interviewer  Guide  —  Part  B) 

G.  Suggestions  for  securing  the  interview  (Pointers  on  the  Interview  Situa- 

tion—  Part  A) 

1.  Paving  the  way 

2.  Credentials 

3.  Contacting  the  respondent 

4.  General  ways  of  securing  cooperation 

5.  Contact  situations  and  how  to  handle 

6.  Noncontact  situations  and  how  to  handle 

H.  Discussion   of   the   interview    (Pointers   on   the   Interview   Situation  — 
Part  B) 

1.  Introduction  and  explanation 

2.  Creating  and  maintaining  rapport 

3.  Type  of  data  sought 

4.  Refusals  and  "don't  know"  answers 

5.  Completing  the  interview 

6.  Use  of  records 

I.    Summary  of  material  covered 

II.  Second  Session 

A.  Question  period  and  discussion  of  first  session 

B.  Question-by-question   review   of   Form   IV   HI    (Interviewer   Guide  — 

PartC-1) 

C.  Question-by-question   review   of   Form    IV   CI    (Interviewer   Guide  — 

Part  C-2) 

D.  Explanation  of  the  accuracy  card  (Interviewer  Guide  —  Part  C-3) 

E.  Discussion  of  the  interviewer  report  form    (Interviewer  Guide  —  Part 

C-4) 

F.  Discussion  of  the  interview  evaluation  form  (Interviewer  Guide  —  Part 

C-5) 

G.  Explanation  of  general  subjects 

1.  Step-by-step  interviewing  procedures 

2.  Spot-checking  of  interviews 

3.  Return  of  carbon  copies  of  advance  letter 

4.  General  field  procedures 
H.  Review  of  list  of  assignments 
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III.  Third  Session 

A.  Question  period  and  discussion  of  second  session 

B.  Explanation  of  role-playing  (Form  IV  HI) 

C.  Explanation  of  role-playing  (Form  IV  CI) 

D.  Final  review 

E.  Presentation  of  pretest  assignments 

F.  Distribution  of  copies  of  take-home  qualifying  examination 

along  with  a  critique  sheet  and  as  he  checked  over  his  mistakes,  the  field 
director  answered  the  interviewer's  questions  and  explained  the  reasons 
for  some  of  the  corrections.  As  each  assignment  was  finished,  the  inter- 
viewers were  given  their  other  completed  assignments  one  at  a  time  until 
all  were  reviewed  and  corrected. 

The  last  part  of  the  meeting  was  devoted  to  a  general  discussion  of 
any  topic  or  problem  which  the  interviewers  felt  they  did  not  under- 
stand. They  were  then  given  two  copies  of  the  "Interviewer  Agreement," 
shown  in  Appendix  B.  One  copy  they  signed  and  returned,  and  the  other 
they  kept  for  their  own  records.  The  main  purpose  of  the  "Interviewer 
Agreement"  was  to  outline  clearly  the  working  arrangements  and  the 
interviewer's  job  responsibilities.  The  agreement  seemed  to  have  added 
value  in  making  the  interviewers  feel  obligated  to  do  the  job.  This  last 
training  session  was  ended  by  handing  out  the  interviewing  assignments. 

All  training  sessions  were  held  in  the  city  where  the  field  operation 
was  being  carried  out  except  in  Study  M4  when  two  cities  readily  acces- 
sible to  the  majority  of  the  candidates  were  used  for  training  sessions. 
The  same  material  was  covered  on  two  consecutive  evenings  in  the  two 
cities;  this  permitted  the  interviewer  to  come  to  either  session. 

As  a  rule,  the  size  of  each  group  was  restricted  to  15  or  16  candidates. 
This  number  was  small  enough  for  individual  questions  to  be  answered, 
and  yet  large  enough  to  reflect  a  variety  of  viewpoints.  Groups  larger 
than  15  or  16  were  divided  into  separate  groups,  with  sessions  for  each 
group. 

Present  at  all  training  sessions  were  the  project  director  or  his  deputy; 
the  field  director,  who  presided  over  the  meeting;  and  field  supervisors, 
if  they  were  being  used  on  that  study.  The  project  director  or  his  deputy 
conducted  the  first  part  of  the  initial  session  and  was  present  at  later 
sessions  to  answer  any  technical  questions  that  might  arise. 

The  day  and  time  of  the  training  sessions  varied  with  each  study. 
They  depended  upon  the  distances  the  candidates  lived  from  the  meeting 
place,  their  other  activities,  when  the  stores  were  open  in  their  areas, 
and  so  forth.  However,  the  participants  generally  preferred  that  the  meet- 
ings be  held  in  the  evenings  about  7:30  during  the  middle  of  the  week, 
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although  in  some  areas  Friday  evenings  or  Saturday  mornings  were 
preferred.  The  meetings  were  planned  to  last  no  longer  than  three  hours, 
with  a  fifteen-minute  break  in  the  middle. 

Compensation  for  the  training  sessions  varied  between  $1.50  and  $1.65 
an  hour,  depending  upon  the  going  rate  in  each  area.  At  the  end  of 
every  meeting  the  candidates  filled  out,  signed,  and  submitted  a  bill  for  the 
time  spent.  No  travel  time  or  mileage  was  paid  for  attendance  at  the 
training  sessions.  Candidates  were  also  paid  for  up  to  two  hours  of  study 
time  on  their  own  and  for  work  on  the  qualifying  examination.33 

No  attempt  was  made  to  evaluate  the  training,  but  some  conclusions 
were  based  on  the  subjective  judgment  of  the  people  involved.  The  use 
of  detailed,  written  instructions  with  specific  examples  was  an  absolute 
necessity;  however,  they  had  to  be  adequately  indexed  for  quick  reference 
in  the  field.  The  value  of  the  qualifying  examination  as  a  training  tool 
was  not  to  be  overlooked.  In  fact,  any  means  of  exposing  the  interviewer 
to  the  written  training  material  was  desirable.  Trial  interviews,  although 
originally  intended  as  a  selection  device,  proved  extremely  effective  in 
getting  interviewers  to  put  training  into  practice.  The  interviewer  did, 
however,  have  to  be  thoroughly  trained  before  attempting  these  trial 
interviews. 

Two  areas  of  training  which  it  was  thought  could  be  of  value  on 
future  financial  studies  were  role-playing  and  the  further  development  of 
the  interviewer's  confidence  in  his  ability  to  obtain  the  personal  financial 
data. 

Inter-wave  Sessions 

The  purpose  of  the  training  sessions  before  each  new  wave  of  inter- 
views was  threefold:  first,  to  maintain  and  improve  the  quality  and 
quantity  of  data  obtained  on  succeeding  waves;  second,  to  stress  good 
interviewing  practices  and  correct  poor  ones;  and  third,  to  train  inter- 
viewers on  any  new  procedures,  experiments,  or  forms  to  be  used  on  the 
current  wave.  As  a  rule,  these  sessions  consisted  of  just  one  approximately 
three-hour  meeting,  although  on  several  occasions  when  a  fairly  large 
amount  of  material  had  to  be  covered,  two  meetings  were  held. 

Each  inter-wave  training  session  was  scheduled  one  week  before  the 
start  of  field  operations.  At  this  meeting,  the  response  and  refusal  rates  of 
the  previous  wave  were  reported,  since  interviewers  had  expressed  interest 
in  knowing  how  they  had  done.  Also  presented  were  statistics  on  the 
quality  of  the  work  received,  number  of  blanks,  number  of  errors,  per- 
centage of  records  used,  and  so  forth.  This  was  a  logical  lead-in  to  a 
discussion  of  how  these  areas  could  be  improved. 

33  See  Chapter  V  for  details  on  interviewer  compensation. 
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The  first  part  of  the  meeting  focused,  therefore,  on  giving  the  inter- 
viewers additional  instruction  in  areas  of  poor  performance.  Also  at  this 
time,  there  was  a  discussion  and  review  of  the  purpose  and  objectives  of 
the  study  to  reinforce  the  interviewers'  confidence  in  the  study  and  give 
them  additional  material  to  overcome  potential  refusals  on  the  present 
wave.  Since  panel  members  tended  to  forget  why  they  were  being  inter- 
viewed and  what  was  being  done  with  the  data,  the  interviewers  had  to 
be  prepared  to  answer  such  questions  again,  in  order  to  reduce  fears  and 
skepticism  about  the  authenticity  of  the  study,  the  use  of  the  data,  or  the 
confidential  treatment  of  answers.  The  importance  of  retaining  each 
respondent  was  emphasized. 

Next  to  be  covered  were  the  different  aspects  of  the  present  wave  of 
interviews,  the  experiments  being  used,  a  question-by-question  review  of 
the  questionnaires,  and  any  changes  in  field  procedures  from  those  of  the 
previous  wave.  The  discussion  of  each  question  included  its  purpose, 
possible  answers,  problems  that  might  arise,  and  methods  of  solving  these 
problems.  An  attempt  was  made  at  this  point  to  relate  the  answers  to 
later  data  analysis,  showing  the  need  for  careful  field  work  and  the 
importance  of  complete  and  accurate  data.  For  each  wave,  the  inter- 
viewers were  given  new  instruction  manuals  incorporating  the  procedures 
to  be  used  on  that  wave.  Verbal  presentation  supplemented  the  written 
manual  and  was  also  used  to  clear  up  questions  raised  in  the  meeting. 

On  the  farm  study  (M4),  some  additional  training  procedures  were 
used.  First,  a  copy  of  the  new  questionnaire  for  that  wave  and  written 
instructions  were  included  with  the  letter  to  the  interviewer  informing 
him  of  the  training  session.  At  the  meeting,  little  time  was  spent  in 
reviewing  the  questionnaire.  Instead,  each  interviewer  was  given  a  copy 
of  the  new  questionnaire  with  fictitious  feedback  information  filled  in.34 
The  field  director  then  gave  answers  to  be  filled  in  for  that  wave  as  if  he 
were  the  respondent.  Completed  questionnaires  were  collected  and  taken 
back  to  the  office  for  detailed  editing.  A  record  was  kept  of  all  mistakes, 
blanks,  and  so  forth,  and  memos  were  sent  to  all  the  interviewers  pointing 
out  errors  and  suggesting  ways  to  improve  the  quality  of  their  work.  The 
filled-in  fictitious  questionnaires  were  returned  with  the  memos  to  the 
interviewers  so  they  could  see  their  mistakes. 

At  the  end  of  this  review  session,  the  interviewers  were  given  four  or 
five   assignments   as   a   start.     These   had   to   be   completed   and   edited 

34  For  the  regular  assignments,  each  interviewer  would  receive  for  each  panel 
member  information  from  the  preceding  interview.  This  feedback  information  was 
filled  in  on  the  new  questionnaire  form  and  consisted  of  holdings  reported  previ- 
ously and  names  of  institutions.  The  interviewer  was  to  obtain  the  current  balance 
in  the  account  (not  filled  in  from  previous  wave)  and  bring  other  details  up  to  date 
where  relevant. 
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thoroughly  by  the  office  before  more  assignments  were  given.  In  this  way, 
many  mistakes  were  caught  early  in  the  wave,  before  they  became  com- 
mon practice. 

On  such  a  panel-type  operation,  it  appeared  that  the  inter-wave 
meetings  contributed  much  to  obtaining  accurate  and  complete  data. 
This  was  especially  true  when  there  was  a  time  lapse  of  a  few  months 
between  waves. 

End-wave  Meetings 

After  the  completion  of  each  wave,  interviewers  were  requested  to 
attend  an  end-of-the-wave  meeting,  which  had  three  purposes:  to  clear 
up  any  field  problems  remaining  from  the  wave  just  completed;  to 
determine  which  areas  of  the  interview  and  the  questionnaire  gave  the 
most  trouble  and  how  these  situations  were  handled;  and  to  improve 
interviewer  morale  through  discussion  of  common  problems  and  the 
sharing  of  experiences.  An  additional  purpose  was  to  lay  the  ground- 
work for  the  next  wave  of  interviews  to  be  conducted  approximately  three 
months  later. 

The  meetings  were  held  not  later  than  one  week  from  the  time  the  last 
interview  was  completed  for  that  wave,  while  experiences  were  still  fresh 
in  the  interviewers'  minds.  A  permissive  atmosphere  was  maintained,  so 
the  interviewers  would  talk  openly  about  their  own  personal  interviewing 
problems.  These  individual  problems  often  proved  to  be  general  problems. 

In  contrast  to  the  structured  training  session  at  the  start  of  the  wave, 
the  end-wave  meeting  took  the  form  of  open  discussion.  Control  was 
maintained  by  the  field  director,  who  presided  at  these  meetings.  In 
addition,  a  staff  analyst  was  also  present  to  record  weak  areas  of  the  just- 
completed  wave  and  to  use  these  experiences  to  avoid  mistakes  in  design- 
ing the  questionnaire  and  field  procedures  for  the  following  wave. 

The  end-wave  meeting  was  especially  useful  for  clearing  up  any  re- 
maining details  of  the  just-completed  wave.  Problems  of  completed  inter- 
views with  blanks,  unclear  answers,  and  so  forth  could  be  discussed  with 
the  interviewers.  The  interviewers  turned  in  all  remaining  assignments 
which  were  being  held  in  the  field,  often  with  the  hope  of  obtaining 
interviews  with  panel  members  so  far  not  available,  temporarily  out  of 
town,  or  busy.  At  the  conclusion  of  this  meeting,  the  wave  was  officially 
closed. 

The  final  points  covered  at  the  end-wave  meeting  were  the  tentative 
plans  for  the  following  wave.  Subjects  included  were  the  approximate 
date  for  the  start  of  interviewing,  who  would  be  interviewed,  (all  panel 
members  or  only  half  of  the  panel  members),  the  approach  to  be  used, 
and  other  known  experiments  and  procedures  to  be  followed.   The  inter- 
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viewers  were  therefore  likely  to  feel  more  involved  with  the  study,  since 
they  knew  what  to  expect  on  the  next  wave. 

Of  all  the  end-wave  meetings,  the  last  one  at  the  end  of  each  study 
proved  to  be  especially  rewarding.  The  interviewers  could  then  feel  free 
to  talk  about  aspects  of  the  study  which  they  may  have  suppressed  in 
order  to  maintain  good  job  relations  during  the  study.  Some  of  the  more 
valuable  points  mentioned  by  the  interviewers  at  the  last  meetings  follow. 

Study  M2 

1.  Importance  of  interviewers'  knowledge  of  financial  terms.  Possi- 
bility of  having  separate  briefing  sessions  with  one  session  on  review  of 
terms. 

2.  Usefulness  of  a  mid-wave  meeting  to  stimulate  morale,  especially 
in  the  early  waves  of  an  operation. 

3.  Completion  of  one  or  two  assignments  by  an  interviewer  and  no 
more  until  these  have  been  checked  by  the  supervisor. 

4.  Notification  of  interviewers  as  to  when  to  expect  pay  checks,  and 
promptness  in  paying  them  on  that  date. 

Study  M3 

1.  Helpfulness  of  more  instructions  placed  right  on  the  questionnaire 
forms,  such  as  instructions  to  cover  all  members  of  the  savings  unit  and 
to  probe  for  additional  holdings. 

2.  Desirability  of  more  attitudinal  questions,  since  they  were  extremely 
useful  in  maintaining  respondent  rapport. 

3.  Possibility  of  talking  with  interviewers  individually  to  solve  personal 
problems  instead  of  having  mid-wave  meetings. 

4.  Appreciation  of  the  initial  set  of  written  instructions  which  proved 
to  be  highly  useful  even  though  they  seemed  overwhelming  at  first. 

Study  M4 

1.  Desirability  of  having  the  interviewers  in  a  future  farm  study 
interview  an  actual  farmer  during  the  initial  training  sessions. 

2.  Possible  usefulness  of  advance  training  on  income  tax  forms  and 
Farm  Bureau  Management  Records. 

3.  Damage  of  some  attitudinal  questions  to  rapport. 

4.  Resolution  of  possible  personality  conflicts  by  having  each  inter- 
viewer rate  the  cooperativeness  of  his  panel  members  on  the  first  wave 
and  switching  uncooperative  members  to  other  interviewers  on  the  next 
wave. 

At  the  last  meeting  of  Study  M4  interviewers,  an  additional  method 
of  obtaining  information  from  them  was  used.  Enclosed  with  the  letter 
sent  to  the  interviewers  informing  them  of  that  meeting  was  a  list  of 
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questions  about  certain  aspects  of  the  study.  The  interviewers  were  asked 
to  bring  their  answers  with  them  to  the  meeting.  In  this  way,  they 
had  a  chance  to  give  more  thought  to  their  answers  and  to  suggest  new 
ideas. 

Recommended  Training  Procedures 

The  training  procedures  described  on  the  preceding  pages  appeared 
to  fulfill  all  the  necessary  requirements.  Indeed,  questionnaires  distributed 
to  the  interviewers  at  the  end  of  the  initial  sessions  in  Studies  M3  and  M4, 
and  filled  in  anonymously,  indicated  general  satisfaction  with  the  pro- 
cedures and  with  the  time  allotted  to  each  topic.  The  only  criticism  was 
that  some  of  the  interviewers  felt  that  too  much  time  had  been  wasted 
by  letting  interviewers  talk  about  unimportant  subjects  and  personal 
experiences.  Nevertheless,  it  was  felt  desirable  to  allow  considerable  dis- 
cussion of  this  sort  rather  than  to  cut  off  further  questions. 

The  dummy  interviews,  the  qualifying  take-home  examination,  and 
the  practice  completion  of  questionnaires  at  the  later  wave  meetings 
appeared  to  have  been  particularly  useful  devices  for  training  the  inter- 
viewers and  for  clarifying  the  type  of  information  being  sought.  The 
presence  at  all  sessions  of  a  project  analyst,  in  some  cases  the  project 
director,  also  appeared  to  have  been  highly  worth  while,  serving  to  convey 
to  the  interviewers  more  of  a  feeling  of  the  importance  of  the  study  and 
at  the  same  time  making  it  possible  to  answer  any  technical  questions 
that  arose. 

On  the  panel  aspects  of  the  operation,  the  inter-wave  training  sessions 
seemed  to  be  essential.  Such  meetings  not  only  helped  make  sure  that 
the  interviewers  were  fully  cognizant  of  changes  in  the  questionnaire,  but 
had  also  not  forgotten  what  the  survey  was  all  about  and  how  to  handle 
particular  situations  as  they  occurred.  These  meetings  also  served  to 
maintain  interviewer  morale,  for  which  purpose  it  was  found  of  par- 
ticular value  to  review  the  experiences  on  the  previous  wave  and  to 
point  out  errors  and  omissions  that  might  have  been  made  at  that  time. 

The  end-wave  meetings  at  the  completion  of  each  wave  also  proved 
to  be  worth  while.  Besides  giving  the  interviewers  an  indication  of  appre- 
ciation for  their  services,  these  meetings  were  highly  useful  as  a  basis  for 
designing  the  questionnaire  for  the  following  wave  and  for  avoiding 
earlier  mistakes.  However,  such  meetings  were  not  treated  as  a  substitute 
for  a  pre-wave  meeting  to  review  the  new  questionnaire  and  operating 
procedures. 

Some  means  of  determining  the  effectiveness  of  training  came  from 
three  distinct  areas.  The  response  and  pick-up  rates  comprised  the  first 
of  these.    The  improvement  in  training  procedures  from  wave  to  wave 
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was  to  some  degree  reflected  in  these  rates.  However,  many  other  vari- 
ables were  operative  which  may  have  accounted  in  part  for  the  improved 
response  and  contact  rates.  Since  each  study  was  slightly  different,  it  was 
not  possible  to  compare  response  and  pick-up  rates  between  studies. 

The  second  measure  of  effectiveness  was  to  be  found  in  the  amount  of 
supervision  required.  As  training  procedures  improved  from  wave  to 
wave  and  from  study  to  study,  the  need  for  supervision  decreased ;  that  is, 
interviewers  tended  to  become  their  own  supervisors  and  catch  their  own 
errors. 

The  third  evaluation  of  training  effectiveness  came  from  the  inter- 
viewers themselves.  The  interviewers'  reaction  to  the  training  was  that  it 
was  adequate  to  good.  The  few  complaints  about  certain  aspects  of 
training  were  resolved  with  improvements  in  the  training  as  the  project 
progressed. 

Needless  to  say,  these  recommended  training  procedures  need  to  be 
tested  under  controlled  experiments  to  establish  their  true  value.  At  the 
same  time,  other  possible  training  procedures  should  be  investigated  to 
determine  the  best  training  to  employ  in  future  financial  studies. 


V.  INTERVIEWER  SUPERVISION 


The  purpose  of  interviewer  supervision  on  a  personal  interview  study 
is  to  see  that  complete  and  accurate  information  is  obtained  from  all 
respondents.  There  are  three  functions  involved.  The  first  is  to  see  that 
written  and  verbal  instructions  to  interviewers  are  carried  out  in  a  manner 
consistent  with  the  original  plan  of  the  study.  Interpreting  these  instruc- 
tions for  handling  specific,  unusual  problems  is  included  in  this  function. 

The  second  function  of  supervision  is  to  identify  interviewers  who  are 
not  meeting  the  standards  of  completeness  and  accuracy  set  for  the  study. 
The  identification  of  these  interviewers  on  the  basis  of  their  performance 
has  a  number  of  uses.  First,  more  effective  interviewers  can  be  selected 
for  succeeding  waves  on  a  panel  operation.  Second,  interviewing  organi- 
zations can  improve  the  effectiveness  of  their  permanent  staff  of  inter- 
viewers. Third,  it  may  prove  feasible  to  set  up  a  sequential  analysis  to 
identify  ineffective  interviewers  as  interviewing  progresses  on  a  single 
interview  study.35  Ineffective  interviewers  can  then  receive  further  train- 
ing or  can  be  dismissed.  Fourth,  when  trainees  are  not  formally  hired 
until  interviewing  performance  on  pilot  studies  or  pretests  is  evaluated, 
it  is  possible  to  hire  only  the  more  effective  trainees. 

The  final  function  of  supervision  is  maintaining  good  interviewer 
morale.  This  function  is  interrelated  with  the  other  two,  and  all  are 
necessary  for  effective  supervision. 

This  chapter  will  cover  these  three  functions  in  the  following  manner. 
The  first  part  will  review  the  procedures  of  supervision  employed  on 
Study  M3  and  will  present  a  quantitative  method  for  identifying  effective 
interviewers  on  the  basis  of  interviewing  performance.  The  second  part 
will  cover  those  aspects  of  supervision  on  the  other  project  studies,  which 
differed  from  the  supervision  on  Study  M3.  Next,  interviewer  and  super- 
visor compensation  used  on  the  project  studies  will  be  discussed.  A  sum- 
mary of  the  findings  and  recommendations  for  the  future  will  comprise 
the  last  part  of  this  chapter. 

35  A  similar  type  of  analysis,  though  one  which  is  not  sequential,  is  proposed 
by  W.  E.  Deming  in  his  book,  Sample  Design  in  Business  Research  (New  York: 
Wiley,  1960),  p.  250. 
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Supervision  in  Study  M3 

A  detailed  discussion  of  the  procedures  of  supervision  employed  on 
Study  M3  will  be  presented.  Unless  otherwise  indicated,  similar  pro- 
cedures and  practices  were  employed  on  the  other  project  studies.  Study 
M3  was  selected  because  validation  data  were  available  which  facilitated 
a  more  rigorous  evaluation  of  interviewer  performance.  In  addition, 
Study  M3  culminated  the  work  in  metropolitan  areas  since,  Study  M4 
was  conducted  in  a  farm  area. 

Selecting  the  Supervisor 

The  function  of  a  field  supervisor  in  Study  M3  was  limited  to  editing 
questionnaires  and  resolving  questions  and  problems  of  the  interviewers. 
This  was  done  because  the  location  of  the  sample  area  was  relatively  close 
to  the  project  headquarters,  enabling  the  field  director  to  maintain  close 
touch  with  the  interviewers. 

Two  supervisors,  hereafter  referred  to  as  editors,  were  hired  to  help 
with  the  field  supervision  of  the  19  interviewers.  One  editor  was  a  36- 
year-old  woman  with  considerable  market  research  interviewing  and 
supervisory  experience.  The  other  editor  was  a  42-year-old  woman  with 
a  full-time  job,  a  business  college  background  and  also  considerable  inter- 
viewing and  supervisory  experience.  The  latter  editor  did  not  do  any 
interviewing  on  Study  M3,  whereas  the  former  interviewed  on  all  five 
waves.  One  of  the  principal  experiments  of  the  study  consisted  of  testing 
a  highly  structured  approach  on  half  of  the  sample  and  a  highly  un- 
structured approach  on  the  other  half  of  the  sample.  Interviewers  were 
given  both  types  of  assignments  and  each  editor  was  trained  to  cope  with 
the  problems  arising  from  each  approach. 

Interview  Assignments  and  Reassignments 

Interview  assignments  were  sent  from  the  project  office  directly  to  the 
interviewers  in  three  separate  mailings,  at  one-week  intervals.  To  facili- 
tate supervision,  lists  of  assignments  were  sent  to  the  editors  who  urged 
the  interviewers  to  make  personal  contact  at  these  sample  addresses  within 
a  few  days.  The  purpose  of  this  quick  contact  was  to  optimize  the  value 
of  the  advance  letter  sent  to  all  sample  members. 

On  the  first  wave,  all  noncontacts  and  refusals  were  reassigned  by  the 
editors  to  different  interviewers.  This  procedure  resulted  in  obtaining 
additional  interviews,  thereby  increasing  the  response  and  contact  rates. 
Refusals  and  noncontacts  were  not  reassigned  on  later  waves  unless  there 
appeared  to  be  a  possibility  that  a  new  interviewer  could  obtain  an  inter- 
view.  The  editors  also  reassigned  sample  members  if  an  interviewer  quit, 
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was  dismissed,  or  could  not  complete  his  assignments  by  the  deadline 
date  for  that  wave. 

Training  and  Retraining  of  Interviewers 

Interviewers  were  trained  primarily  by  the  field  director.    However, 
before  each  training  session,  the  editors  were  briefed  on  the  content  of  the 

Figure  8.  Reminder  Notes  for  Wave  2 :    Study  M3 

Checklist  for  Handling  Each  Assignment 

A.  Before  the  Interview 

1.  Review  Wave  1  IRF. 

a.  When,  where,  and  who  (best  time,  place,  and  person  to  interview). 

b.  Attitude  of  the  panel  member. 

c.  "Summary  Comments"  and  "Interviewer's  Notes." 

2.  Study  "Problems  and  Comments"  sheet  if  with  assignment. 

3.  Review  "Background  Form"  (III-2). 

a.  Date  of  last  interview. 

b.  Family  composition. 

4.  Look  over  the  "Recording  Form"  (III  HA2,  III  HS2,  III  CA2,  or  III 
CS2). 

a.  Approach  being  used. 

b.  Note  if  there  is  any  missing  information. 

5.  Read  Wave  2  advance  letter. 

a.  Correct  name  and  address  if  in  error. 

b.  Note  any  special  circumstances  in  letter. 

6.  Have  extra  cards  A,  B,  and  C  with  you. 

B.  During  the  Interview 

1.  Follow  instructions  for  the  approach  being  used. 

2.  At  end  of  interview  review  information  given,  filling  in  all  blanks. 

C.  After  the  Interview 

1.  Correct  name  and  address  on  carbon  of  advance  letter  and  mail  directly 
to  Champaign. 

2.  Edit  thoroughly  Wave  2  IRF,  the  Background  Form,  and  the  Recording 
Form.  If  any  information  is  missing  or  not  clear,  correct  before  sending 
to  the  editor. 

3.  Send  to  the  editor: 

a.  III-IRF-2. 

b.  Background  Form  (III-2). 

c.  Recording  Form  (III  HA2,  III  HS2,  III  CA2,  or  III  CS2). 

d.  Wave  1  IRF. 

e.  Problems  and  comments  sheet,  if  any  is  included. 

4.  Record  of  your  bill : 

a.  Date  interview  completed. 

b.  Interviewee  number. 

c.  Interviewee  initials. 

d.  Rate  from  lower  right  hand  corner  of  Wave  2  IRF. 


Interviewer  Supervision  69 

session  and  their  reactions  to  proposed  procedures  were  secured.  In  addi- 
tion, both  editors  had  to  train  interviewers  who  could  not  attend  some 
of  the  later  wave  sessions,  and  in  some  instances  they  had  to  give  further 
training  to  interviewers  who  were  having  particular  problems. 

As  training  improved  on  each  wave,  a  continuing  effort  was  made  to 
have  each  interviewer  be  his  own  editor.  For  example,  on  Wave  2  the 
interviewers  were  given  a  one-page  instructional  sheet  called  "Reminder 
Notes,"  which  was  a  checklist  for  handling  each  assignment  (see  Figure 
8) .  The  editors'  work  was  greatly  reduced  when  the  interviewers  followed 
the  checklist. 

Returning  Completed  Work 

Completed  assignments,  regardless  of  whether  they  were  interviews, 
refusals,  or  noncontacts,  were  to  be  sent  to  the  editors  by  9 :  00  a.m.  on  the 
day  following  their  completion.  This  compelled  the  interviewers  to  edit 
and  complete  assignments  before  the  details  were  forgotten.  This  also 
made  it  possible  for  the  editors  to  reassign  noninterviews  and  for  the  office 
to  send  thank-you  letters  soon  after  the  interviews  were  completed.  To 
facilitate  mailing,  stamped  envelopes  addressed  to  the  editors  were  en- 
closed with  the  assignments  when  they  were  given  to  the  interviewers. 
If  completed  assignments,  upon  receipt  by  the  editors,  had  no  editing 
corrections,  they  could  be  sent  to  the  project  office.  If  there  were  minor 
errors,  these  were  usually  corrected  through  phone  calls  to  the  inter- 
viewers. If  major  omissions  of  information  were  involved,  the  editors 
returned  the  assignments  to  the  interviewers  for  re-editing  or  for  reinter- 
views  with  the  sample  members  to  obtain  the  missing  data.  These  proce- 
dures, it  was  felt,  forced  the  interviewers  to  edit  their  own  questionnaires, 
and  this  reduced  the  frequency  with  which  incomplete  questionnaires  were 
sent  to  the  editors. 

On  Wave  5,  the  interviewers  returned  all  completed  assignments  di- 
rectly to  the  project  office  rather  than  to  the  field  editors.  On  the  previous 
four  waves,  the  interviewers  had  been  conditioned  to  look  over  their  work 
carefully  before  sending  it  to  the  editors.  On  the  last  wave,  it  was  felt 
that  the  time  schedule  could  be  better  controlled  at  the  office  level,  and 
this  method  proved  very  satisfactory.  In  a  meeting  at  the  end  of  the 
study,  the  interviewers  expressed  the  feeling  that  editors  were  essential 
for  the  first  wave  and  possibly  for  the  second  wave,  but  that  there  was  no 
real  need  for  them  on  subsequent  waves. 

Identifying  Effective  Interviewers 

Five  methods  were  used  in  the  identification  of  good  interviewers: 
( 1 )  determining  the  quality  of  their  work  from  the  evaluations  of  editors 
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in  the  field  and  editors  and  coders  in  the  office;  (2)  evaluating  their  work 
through  the  use  of  "The  Editor's  Weekly  Report";  (3)  spot  checking  to 
estimate  the  quantity  and  quality  of  their  work;  (4)  changing  inter- 
viewers on  a  later  wave;  and  (5)  employing  a  quantitative  method.  The 
validation  data  were  used  to  evaluate  these  five  means  of  identifying 
effective  interviewers. 

Editing 

Editing  consisted  of  a  detailed  check  of  the  questionnaire  and  of  the 
Interviewer  Report  Form  (Appendix  C)  for  blanks,  unclear  answers, 
obvious  inconsistencies,  and  unclear  explanations  of  unusual  situations. 
The  editors  noted  excessive  "don't  know"  answers  and  question  refusals 
in  the  interviewers'  work  and  discussed  this  problem  with  them.  Formal 
records  of  each  interviewer's  performance  were  prepared,  based  on  office 
coding  and  editing.  The  field  editors  were  informed  of  office  tabulations 
so  they  could  watch  or  retrain  poor  interviewers.  On  each  succeeding 
wave,  this  record  of  errors  was  used  as  a  basis  for  interviewer  training. 
At  the  end  of  each  wave  the  interviewers  were  given  a  report  of  their 
performance  on  that  wave  as  well  as  a  summary  of  the  performance  of 
all  interviewers. 

Editor's  Weekly  Report 

To  identify  interviewers  who  were  procrastinating  in  completing  as- 
signments, the  editors  were  required  to  prepare  and  send  to  the  office  the 
"Editor's  Weekly  Report"  (see  Figure  9).  As  its  name  implies,  this 
report  was  sent  weekly  to  the  field  director,  and  gave  a  quantitative 
picture  of  the  field  operations  as  of  midnight  Saturday.  This  form  was 
useful  in  spotting  inactive  interviewers  and  in  expediting  completion  of 
the  field  work. 

Spot  Checking 

On  Wave  3,  a  spot-check  procedure  was  instituted  on  the  basis  of 
validation  material.  If  the  known  financial  holding  was  listed  on  the 
questionnaire,  it  was  assumed  that  the  interviewer  had  made  the  inter- 
view and  no  check  was  made.  If  the  known  holding  in  the  financial 
institution  was  not  reported,  a  post  card  was  sent  to  the  respondent 
ostensibly  seeking  information  on  the  length  of  the  interview  and  on  the 
interviewer's  characteristics.  Such  a  post  card  is  shown  in  Figure  10. 
A  total  of  80  cards  were  sent  and  62.5  percent  were  returned.  All  of  the 
sample  members  indicated  they  were  interviewed,  although  two  gave 
their  interviewers  a  poor  rating.  Of  the  50  returned,  23  gave  additional 
written  comments,  most  of  which  were  favorable.  A  sample  was  picked 
from  the  30  who  did  not  return  the  post  card  and  approximately  15 
phone  calls  were  made  to  check  if  the  interview  had  actually  occurred. 
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Figure  10.   Spot-check  Post  Card:    Study  M3 
The  interview  on took  No 

]  less  than  15  minutes  ]  35  to  44  minutes 

]   15  to  24  minutes  ]  45  to  59  minutes 

1  25  to  34  minutes  |  ]  one  hour  or  more 

The  interviewer  impressed  me  as  follows: 

|~1  Nice  person  to  talk  to  \~\  Aggressive,  pushy 

2  Quiet  ]  Not  very  bright 
l2  Intelligent  Q  Overly  friendly 
J  Neat,  well-dressed  |   ]  Talkative 

! '  ]  Tactful  i  ]  Poor  appearance 

D- 

Comments . 

No  falsification  was  uncovered  by  these  calls.  It  was  felt  that  the  primary 
value  of  the  use  of  the  post  cards  was  in  forestalling  interviewer  cheating, 
as  interviewers  were  informed  during  the  initial  training  sessions  that  this 
type  of  spot  check  would  be  made.36 

Changing  Interviewers 

At  the  initial  training  sessions,  interviewers  were  informed  that  half 
of  their  panel  members  would  be  reassigned  to  other  interviewers  on  the 
third  wave.  Although  the  main  purpose  of  this  experiment  was  to  explore 
the  possibility  that  the  new  interviewer  might  pick  up  information  missed 
by  the  earlier  one,  the  interviewers  were  undoubtedly  aware  that  this 
procedure  would  also  serve  to  uncover  falsified  interviews.  In  addition, 
knowing  that  this  procedure  was  to  be  used  may  well  have  motivated  the 
interviewers  to  do  a  more  thorough  job  of  interviewing.  No  appreciable 
difference  in  response  and  pick-up  rates  was  observed  between  panel 
members  who  were  reassigned  and  those  who  were  not. 

Using  a  Quantitative  Method 

For  Study  M3,  the  pick-up  rate  (response  rate  times  the  completeness 
rate)  was  employed  as  the  criterion  of  interviewer  effectiveness.  Where 
such  validated  criteria  are  not  available,  other  variables  must  be  employed. 
The  present  analysis  evaluates  the  extent  to  which  these  other  variables 
could  have  been  used  to  identify  interviewers  who  had  a  high  pick-up 
rate  on  Study  M3. 

36  Based  on  a  recent  experiment,  a  question  may  be  raised  as  to  the  value  of 
postcards  to  detect  interviewer  cheating.  "In  this  experiment,  ten  people  selected 
at  random  from  the  Chicago  telephone  directory  were  sent  thank-you  letters,  on 
University  of  Illinois  stationery,  for  granting  an  interview  on  their  finances.  Al- 
though these  people  were  not  actually  interviewed,  five  returned  the  postcard  indi- 
cating the  length  of  the  interview  and  their  opinion  of  the  interviewer!"  Robert 
Ferber,  The  Measurement  and  Control  of  Errors:  Savings  Data,  op.  cit.,  Chap- 
ter 13. 


Interviewer  Supervision  73 

The  following  variables  may  be  calculated  for  most  survey  field 
studies : 

1.  Response  rate. 

2.  Percentage  of  ambiguous  (unclear)  answers. 

3.  Percentage  of  refusals  of  specific  information. 

4.  Percentage  of  respondents  using  records. 

5.  Percentage  of  questionnaires  evaluated  as  "less  than  fully  complete" 
(interviewer  evaluation) . 

6.  Average  length  of  interview. 

7.  Variability  in  length  of  interview. 

8.  Percentage  of  interviews  in  which  husband  and  wife  both  par- 
ticipated. 

9.  Mean  number  of  specific  holdings  reported. 

The  relationship  between  each  of  these  variables  and  each  inter- 
viewer's pick-up  rate  will  be  evaluated  first.  Two  types  of  statistical 
analysis  will  be  employed.  The  first  is  correlation  analysis,  using  the 
coefficient  of  determination  corrected  for  degrees  of  freedom  (r2).37  The 
other  test  employed  is  the  Mann- Whitney  U  Test.38  This  is  used  to  test 
the  significance  of  the  difference  between  the  scores  of  one  interviewer 
group  and  those  of  another.  Its  primary  drawback  is  that  the  difference 
between  only  two  groups  of  interviewers  can  be  tested  at  a  time. 

1.  Response  rate.  The  response  rate  is  frequently  employed  as  an  indi- 
cator of  interviewer  effectiveness,39  since  it  is  assumed  that  the  interviewer 
who  does  well  at  the  door  will  also  be  effective  in  the  interview.  Although 
significant,  an  r2  of  only  .18  was  obtained  between  the  pick-up  rate  and 
the  response  rate.  The  direction  of  this  relationship  was  positive.  In 
addition,  an  r2  of  .05  was  obtained  between  response  rates  and  complete- 
ness rates.  This  did  not  differ  significantly  from  zero. 

In  view  of  the  importance  attached  to  the  response  rate  as  a  means 
of  evaluating  interviewer  effectiveness  on  sample  surveys,  these  findings 
suggested  that  a  much  more  vigorous  examination  of  the  relationship  be- 
tween response  rates  and  interviewer  effectiveness  was  needed. 

2.  Percentage  of  ambiguous  answers.  The  most  frequent  type  of 
ambiguous  answer  occurred  when  the  interviewer  left  a  question  blank 

37  Since  the  sample  size  is  the  same  for  the  coefficients  of  determination  pre- 
sented in  this  section,  it  might  be  noted  that  an  r2  of  .121  differs  significantly  from 
zero  at  the  5  percent  level,  and  an  r2  of  .266  is  significantly  different  from  zero  at 
the  1  percent  level.  In  this  section,  all  r2  values  are  corrected  values  and,  if  re- 
ported significant,  are  significant  at  the  5  percent  level  unless  otherwise   stated. 

3S  See  Sidney  Siegel,  Nonpar ametric  Statistics  (New  York:  McGraw-Hill, 
1956),  pp.  116-27. 

39  Donald  M.  Hobart,  Marketing  Research  Practice  (New  York:  Ronald, 
1950),  p.  102. 
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where  some  explanation  was  required.  It  may  have  occurred  because  the 
respondent  did  not  own  an  asset  or  had  "no  change"  in  the  asset  and 
the  interviewer,  under  the  pressure  of  the  interview  situation,  failed  to 
check  the  appropriate  box.  An  ambiguous  answer  may  also  have  occurred 
because  the  interviewer  failed  to  do  adequate  probing.  The  number  of 
ambiguous  answers  in  a  questionnaire  tended  to  indicate  something  of 
the  interviewer's  thoroughness  in  work  habits,  either  in  terms  of  whether 
the  questionnaire  was  edited  or  in  terms  of  the  thoroughness  of  the 
editing. 

Ambiguities  on  four  types  of  holdings  were  counted :  savings  accounts, 
checking  accounts,  life  insurance,  and  debts.  The  number  of  these  as  a 
percentage  of  four  times  the  number  of  interviews  taken  by  an  interviewer 
was  the  criterion  used.   The  average  per  interview  was  2  percent. 

The  corrected  coefficient  of  determination  for  the  percentage  of 
ambiguities  was  zero  for  the  pick-up  rate,  which  suggested  that  the  use 
of  this  variable  to  indicate  over-all  interviewer  effectiveness  was  ques- 
tionable. 

3.  Percentage  of  refusals  on  specific  information.  Some  respondents 
granted  interviews  but  then  refused  to  answer  certain  questions.  A 
"don't  know"  response  was  classified  for  the  purposes  of  this  analysis  as  a 
refusal.  While  in  some  cases  a  "don't  know"  answer  may  have  been 
a  truthful  one,  in  others  it  was  another  way  of  refusing  information.  The 
measure  used  was  the  number  of  refusals  and  "don't  know"  answers 
recorded  by  an  interviewer  under  the  sections  dealing  with  savings  ac- 
counts, checking  accounts,  life  insurance,  and  debts  as  a  percentage  of 
four  times  his  number  of  interviews.  A  corrected  coefficient  of  determina- 
tion of  zero  was  obtained  between  this  measure  and  the  pick-up  rate. 

4.  Percentage  of  respondents  using  records.  Two  measures  of  the  use 
of  records  were  prepared.  One  was  based  on  the  percentage  of  reported 
holdings  where  records  were  used,  including  all  types  of  assets  and  debts 
except  charge  accounts.  The  measure  employed  was  the  percentage  of  an 
interviewer's  respondents  using  records  on  25  percent  or  more  of  their 
reported  holdings.  An  r2  of  .02  was  obtained  between  this  measure  and 
the  pick-up  rate.   This  coefficient  did  not  differ  significantly  from  zero. 

Of  some  interest  was  the  fact  that  an  r2  of  .01  was  obtained  between 
the  use  of  records  and  the  percentage  of  refusals  on  individual  holdings. 
An  r2  of  .01  was  also  obtained  with  the  percentage  of  ambiguous  answers. 

5.  Percentage  of  questionnaires  evaluated  as  "less  than  fully  com- 
plete." The  measure  used  was  the  percentage  of  respondents  whose  re- 
ports of  holdings  of  savings  accounts  were  evaluated  by  the  interviewers  as 
"less  than  fully  complete"  (see  Appendix  C) .  An  r2  for  this  variable  with 
the  pick-up  rate  was  zero. 
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A  corrected  coefficient  of  .33  was  obtained  between  use  of  records  and 
evaluation  of  completeness.  The  regression  coefficient  was  negative.  This 
indicated^  as  would  have  been  expected,  that  the  use  of  records  by  a 
respondent  influenced  the  interviewer's  evaluation  of  respondent  coopera- 
tion. 

6.  Average  length  of  interview.  The  length  of  the  interview  expressed 
in  minutes  was  not  related  to  the  pick-up  rate.  The  length  of  interview, 
however,  did  show  a  significant  adjusted  coefficient  of  .21  with  the  re- 
sponse rate.  The  regression  coefficient  was  negative,  indicating  that 
interviewers  with  high  response  rates  tended  to  take  a  shorter  length  of 
time  in  the  interview.  This  may  have  reflected  the  inclusion  of  less 
cooperative  panel  members  or  basic  differences  in  interviewer  character- 
istics and  techniques. 

7.  Variability  in  length  of  interview.  This  measure  was  prepared  as 
follows.  The  mean  length  of  interview  with  respondents  in  four  different 
size  categories  of  total  financial  holdings  was  computed  for  each  inter- 
view. The  time  required  for  each  interview  was  subtracted  from  the 
average  for  the  size  category  and  squared.  These  squared  deviations 
were  summed  for  each  interviewer  across  all  size-of-holdings  groups.  This 
can  be  summarized  as 


EE^-za2, 


i=l  j=l 

where  X\j  was  the  ;th  interview  in  the  ith  size-of-holdings  category,  n  was 
the  number  of  interviews  taken  by  the  interviewer  in  the  ith  size-of- 
holdings  group,  and  m  was  4. 

The  coefficient  of  determination  for  this  variable  with  pick-up  rate 
did  not  differ  significantly  from  zero.  However,  a  significant  r2  of  .16 
was  obtained  between  this  measure  and  the  percentage  of  ambiguous 
answers. 

8.  Percentage  of  interviews  in  which  husband  and  wife  both  partic- 
ipated. Interviewers  were  instructed  to  obtain  joint  interviews  with 
husband  and  wife  whenever  possible.  It  was  felt  that  more  accurate 
reporting  would  be  obtained  in  this  type  of  interview.  The  findings  from 
the  study  indicated  that  this  was,  in  fact,  true.  Those  interviews  in  which 
both  husband  and  wife  participated  were  significantly  more  accurate  than 
those  in  which  only  one  member  of  the  savings  unit  participated. 

The  percentage  of  interviews  in  which  both  husband  and  wife  par- 
ticipated was  prepared  for  each  interviewer.  However,  an  r2  of  zero  was 
obtained  between  this  measure  and  interviewer  pick-up  rate. 

Interviewer  response  rate  was  significantly  related  to  the  percentage 
of  interviews  in  which  both  husband  and  wife  participated.  The  r2  was  .22 
and  the  regression  coefficient  was  positive. 
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9.  Mean  number  of  specific  holdings  reported.  Since  one  character- 
istic of  the  more  effective  interviewers  was  their  ability  to  obtain  mention 
of  holdings  more  frequently  than  the  less  effective  ones,  the  mean  number 
of  holdings  reported  by  their  respondents  may  have  been  larger.  The 
holdings  used  were  those  with  a  minimum  of  respondent  variability  where 
the  number  held  by  a  respondent  was  limited  and  where  most  respondents 
owned  holdings.  Three  types  of  holdings  were  included:  checking 
accounts,  savings  accounts,  and  non-charge-account  debts.  An  adjusted 
coefficient  of  determination  of  zero  was  obtained  for  each  of  these  hold- 
ings, yet  the  mean  number  of  each  of  these  holdings  reported  by  respond- 
ents of  the  more  effective  interviewer  groups  was  larger,  though  not 
significantly  larger,  than  that  for  the  less  effective  interviewer  groups. 

The  criteria  of  interviewer  effectiveness  frequently  employed  on  sur- 
veys for  which  no  validation  data  are  available  are  response  rates,  per- 
centage of  ambiguous  answers,  and  percentage  of  refusals  of  specific 
information.   The  following  regression  was  obtained  using  these  variables. 

Pick-up  rate  =  8.7  +  .621*1  -  .285z2  -  .281z3 

(.245)      (.557)       (.253) 
R2  =     .18 
Su  =  8.6 

In  this  equation  Ziwas  the  response  rate,  z2  was  the  percentage  of  ambig- 
uous answers,  z3  was  the  percentage  of  refusals  of  specific  information, 
and  8.7  was  the  constant  term.  The  number  in  parentheses  below  each 
net  regression  coefficient  is  the  standard  error  of  the  coefficient.  The 
R2  is  the  corrected  coefficient  of  multiple  determination,  and  the  Su  is  the 
corrected  standard  error  of  the  estimate. 

These  variables  explained  about  18  percent  of  the  variance  in  the 
pick-up  rate  of  the  interviewers,  which  was  significantly  different  from 
zero.  The  net  regression  coefficient  for  response  rates  was  the  only  one 
which  differed  significantly  from  zero.  The  reliability  of  estimates  ob- 
tained with  this  equation,  as  reflected  by  the  standard  error  of  the  esti- 
mate, was  low  when  compared  with  that  obtained  with  subsequent  equa- 
tions employing  other  variables. 

If  the  interviewers  were  divided  into  thirds  on  the  basis  of  their  pre- 
dicted pick-up  rates,  the  actual  pick-up  rates  for  these  groups  could  be 
compared.   The  results  of  such  a  grouping  are  presented  below. 

Predicted                                                   Completeness  Pick-up  Number  of 

pick-up  rate                                                          rate                     rate  interviewers 

Highest  third 67%                 54%  6 

Middle  third 67                     48  6 

Lowest  third 72                      47  6 

All  interviewers 68                      50  18 
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While  in  terms  of  the  pick-up  rate  there  was  some  tendency  to  select 
the  most  effective  interviewers,  this  equation  did  not  distinguish  among 
the  lower  two-thirds  of  the  interviewers.  In  fact,  there  was  a  tendency 
for  the  equation  to  eliminate  the  interviewers  who  would  be  more  effec- 
tive in  the  interview  as  indicated  by  the  higher  completeness  rate  of  the 
lowest  third  of  the  interviewers. 

An  interviewer's  performance  in  the  interview  once  he  is  past  the  door 
is  frequently  evaluated  by  the  percentage  of  ambiguous  answers  and  the 
percentage  of  refusals  of  specific  data.  The  measure  of  interviewer  effec- 
tiveness in  the  interview  employed  in  this  study  was  the  completeness  rate. 
To  what  extent  could  the  percentage  of  ambiguous  answers  (z2)  and  of 
refusals  (z3)  be  used  to  identify  the  interviewers  who  were  effective  in  the 
interview?   The  following  equation  was  obtained. 

Completeness  rate  =  74.7  —  A30zo  —  .560z3 

(.663)       (.308) 
R2  =      .08 
Su  =  11.0 

The  small  amount  of  variance  explained  by  these  two  variables  indi- 
cates that  studies  in  which  they  are  used  as  criteria  of  interviewer  effec- 
tiveness are  measuring  something  different  from  what  is  measured  by  the 
completeness  rate  employed  in  the  present  analysis. 

A  number  of  different  combinations  of  the  nine  variables  were  tested 
in  an  attempt  to  explain  the  varying  pick-up  rates  of  interviewers.  Since 
the  response  rate  is  a  major  component  of  the  pick-up  rate,  it  seemed 
logical  to  include  it.  The  following  equation  was  the  best  discovered  in 
terms  of  the  criteria  presented  in  Chapter  III. 

Pick-up  rate  = 

20.6  +  1.118Zl  -  .450*3  +  -165z4  +  .136z6  -  .177z7  +  .231*8 
(.219)       (.193)       (.068)       (.059)       (.054)       (.118) 
R2  =    .63 

"a  —  D.O 

In  this  equation  zx  was  the  response  rate  of  each  interviewer,  z3  was 
the  percentage  refusing  specific  information,  z4  was  the  percentage  using 
records,  z6  was  the  average  length  of  interview,  z7  was  the  variability  in 
length  of  interview,  and  z8  was  the  percentage  of  interviews  with  both 
husband  and  wife. 

The  respective  beta  coefficients  were  as  follows : 

flx  =  1.174,  B3  =  .420,  B%  =  .375,  B6  =  .479,  B7  =  .539,  and  Ba  = 
.379. 

The  correlation  coefficient  was  significantly  different  from  zero  as  was 
each  of  the  net  regression  coefficients.    The  response  rate  stood  out  as 
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being  of  particular  importance  in  selecting  effective  interviewers.    The 
remaining  variables  were  of  about  equal  importance. 

If  the  18  interviewers  were  ranked  on  the  basis  of  their  computed 
pick-up  rate,  the  highest  third  would  have  an  observed  pick-up  rate  18 
percentage  points  higher  than  the  lowest  third.  In  addition,  as  the  fol- 
lowing tabulation  indicates,  the  middle  third  would  have  a  rate  midway 
between  the  two  extreme  groups,  suggesting  a  continuous  differentiation 
of  interviewers  by  this  regression  equation. 

Computed                                   Completeness  Pick-up 

pick-up  rate                                       rate  rate 

Highest  third 78%  59% 

Middle  third 67  50 

Lowest  third 59  41 

All  interviewers 68  50 

Since  the  completeness  rate  of  the  highest  third  is  18  percentage 
points  above  those  of  the  lowest  third,  this  equation  did  identify  inter- 
viewers who  were  more  effective  in  the  interview  as  well  as  at  the  door. 

These  findings  suggest  several  things.  One  is  that  effective  interviewers 
can  be  identified  on  the  basis  of  interviewing  performance,  which  in  turn 
suggests  that  more  reliable  regression  equations  might  form  the  basis  for 
a  sequential  analysis  model  which  would  signal  when  interviewers  need 
additional  training  or  possibly  dismissal.  Such  a  model  could  be  useful 
on  a  one-time  survey  as  well  as  on  a  continuing  survey  operation.  The 
findings  also  suggest  the  need  for  further  research  on  interviewer  effective- 
ness. Since  these  equations  evolved  from  Study  M3  data,  the  findings 
presented  here  would  require  independent  testing  to  establish  their  validity 
and  reliability. 

Interviewer  Morale 

An  important  part  of  the  job  of  supervision  is  establishing  and  main- 
taining a  high  level  of  interviewer  morale.  Clearly,  interviewer. morale  is 
an  important  factor  influencing  the  completeness  and  accuracy  of  infor- 
mation obtained  by  interviewers.  Yet,  because  of  its  subjective  nature, 
it  is  difficult  to  identify  the  dimensions  of  interviewer  morale  and  the 
factors  which  influence  it.  Further  work  on  this  important  problem  is 
needed. 

One  factor  influencing  interviewer  morale  is  the  interviewer's  percep- 
tion of  the  importance  of  the  information  collected.  While  the  importance 
of  the  study  can  be  communicated  by  the  study  and  field  directors,  its 
importance  may  be  indicated  with  equal  or  greater  force  by  the  actions 
of  supervisors  and  the  standards  they  set. 

Interviewer  morale  is  also  influenced  by  the  interviewer's  perception 
of  a  respondent's  willingness  to  report  the  information  sought.    As  was 
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indicated  earlier,  a  function  of  the  training  sessions  in  the  project  studies 
was  to  create  favorable  interviewer  attitudes  toward  interviewing.  The 
supervisor  can  influence  interviewer  attitudes  through  recognition  and 
discussion  of  interviewer  problems.  Close  contact  between  supervisor  and 
interviewer,  where  problems  can  frequently  be  discussed,  can  contribute 
to  favorable  interviewer  attitudes. 

An  interviewer's  attitude  toward  his  ability  to  deal  with  problems 
which  arise  also  influences  morale.  Here  again,  a  part  of  the  supervisor's 
role  can  be  viewed  as  an  extension  of  the  training  program.  Inevitably, 
unforseen  problems  arise  in  the  field.  If  a  supervisor  is  available,  the 
interviewer  can  discuss  a  problem  and  resolve  it  immediately.  As  a  result, 
on  succeeding  interviews  the  interviewer  feels  that  he  is  able  to  handle 
such  problems  if  they  arise.  In  the  project  studies  frequent  letters  and 
phone  calls  from  the  office  were  also  employed  to  give  the  interviewer  the 
feeling  that  he  was  being  kept  up-to-date  and  that  he  had  expert  advice 
available  if  problems  arose. 

As  a  further  means  of  maintaining  a  high  morale,  response  and  con- 
tact rates  were  discussed  at  meetings  during  the  wave  and  at  the  end  of 
the  wave.  In  this  way,  interviewers  could  evaluate  their  work.  At  these 
meetings,  interviewers  were  also  encouraged  to  discuss  their  problems 
openly.  Not  only  were  answers  to  these  problems  worked  out,  but  inter- 
viewers could  see  that  the  other  interviewers  had  similar  problems. 

Aspects  of  Supervision  in  Other  Project  Studies 
Which  Differed  From  Study  M3 

Field  Supervisors  Not  Used  for  Study  M4 

On  Study  M4,  the  farm  study,  all  supervision  was  handled  directly  by 
the  field  director,  and  no  field  editors  or  supervisors  were  hired.  The 
primary  reason  for  this  procedure  was  that  the  sample  members  lived  in 
a  three-county  area  covering  approximately  3,000  square  miles.  As  a 
result,  interviewers  were  scattered  correspondingly  over  a  wide  area  to 
minimize  travel  time  and  expense.  Since  the  sample  area  was  close  to 
the  project  headquarters,  it  was  felt  that  little  saving  would  have  been 
achieved  by  having  supervisors  or  editors  maintain  contact  with  the 
interviewers. 

On  Study  M4,  a  policy  was  adopted  of  giving  assignments  to  the  inter- 
viewers based  on  how  many  assignments  they  were  holding  at  the  time. 
At  no  time  were  the  interviewers  to  have  a  backlog  of  unworked  assign- 
ments. Previous  experience  had  shown  that  interviewers  with  many  un- 
worked assignments  were  likely  either  to  do  a  poor  job  in  order  to  com- 
plete the  interviews  or  not  to  complete  them  by  the  deadline  date. 

To  aid  the  interviewers  in  the  completion  of  their  assignments  and  to 
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reduce  travel  time,  assignments  were  given  by  geographic  areas.  As  had 
been  done  in  the  previous  studies,  a  stamped  post  card  addressed  to  the 
office  was  enclosed  with  each  group  of  assignments  sent  to  the  inter- 
viewers. The  return  of  this  card  notified  the  office  that  the  assignments 
had  been  received.  To  complete  the  card,  interviewers  had  to  look  over 
their  material  and  indicate  on  the  card  what  had  been  received.  This,  it 
was  found,  encouraged  them  to  begin  interviewing  more  quickly  and, 
thus,  to  obtain  the  benefit  of  the  advance  letter.  On  the  later  waves,  after 
rapport  had  been  established,  contacting  the  panel  member  soon  after 
the  advance  letter  had  been  sent  did  not  seem  to  be  as  important  as  it 
had  been  on  earlier  waves. 

On  Waves  4  and  5  a  deadline  date  was  set  for  the  completion  of  each 
assignment  in  order  to  obtain  still  greater  control  of  the  time  schedule. 
For  the  majority  of  the  assignments,  the  deadline  date  was  approximately 
one  week  after  the  assignment  was  sent  to  the  interviewer.  This  allowed 
time  for  the  interviewer  to  get  in  touch  with  the  panel  member,  make  an 
appointment  if  necessary,  and  obtain  the  interview.  If  the  interviewer  was 
not  able  to  complete  an  assignment  by  its  deadline  date,  he  had  to  call 
the  field  director  and  explain  the  delay.  If  the  explanation  was  reason- 
able and  if  there  was  a  possibility  of  an  interview,  the  assignment  was 
left  with  the  interviewer  and  a  new  deadline  date  was  set.  In  a  few 
instances  the  interviewer  was  requested  to  send  the  incomplete  assignment 
to  the  field  office,  and  it  was  then  either  dropped  or  assigned  to  a  new 
interviewer. 

A  close  record  of  deadline  dates  was  maintained  so  that  if  an  inter- 
viewer neglected  to  call  and  explain  why  an  assignment  was  not  com- 
pleted, he  would  be  called  by  the  field  director.  This  procedure  proved 
worth  while  and  was  instrumental  in  getting  the  field  work  done  in  one 
month  compared  with  two  or  more  months  for  previous  waves  and  pre- 
vious studies. 

Return  of  Carbon  Copy  of  Advance  Letters,  Study  M4 

On  Study  M4,  the  interviewers  were  requested  to  return  the  carbon 
copy  of  the  advance  letter  in  a  separate  envelope  as  soon  as  the  assign- 
ment was  completed.  Therefore,  when  a  carbon  copy  was  received,  the 
field  director  knew  that  the  assignment  had  been  completed  and  that  the 
questionnaire  could  be  expected  in  the  mail  within  the  next  day  or  two. 
Interviewers  were  contacted  and  asked  for  an  explanation  if  the  com- 
pleted assignment  was  not  received  within  three  days  after  this  carbon  had 
been  returned. 

Interviewer  Morale  in  Study  M4 

When  interviewers  work  by  themselves  without  close  field  supervision, 
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there  is  a  good  chance  that  morale  will  suffer.  In  Study  M4  interviewers 
were  encouraged  to  get  in  touch  with  the  field  director  at  any  time  of  the 
day  or  night  if  they  had  a  problem.  They,  too,  were  contacted  frequently 
—  at  least  once  a  week  —  by  phone,  by  letter,  or  by  a  personal  visit. 
Care  was  taken  not  to  give  the  interviewers  too  many  assignments  at  one 
time,  and  pressure  to  complete  assignments  was  kept  at  a  reasonable  level. 
The  quantity  and  quality  of  each  interviewer's  work  was  kept  under  close 
observation,  and  if  either  appeared  to  be  deteriorating  the  interviewer  was 
contacted  and  the  problem  discussed  with  him. 

When  supervision  was  handled  by  the  field  director,  tighter  control 
was  possible  even  though  the  interviewers  worked  more  independently. 
The  project  staff  knew  what  was  wanted,  and  controls  were  set  up  so 
that  no  time  was  lost  by  having  the  assignments  go  through  field  super- 
visors who  were  not  so  familiar  with  all  the  needs  of  the  study.  Poor- 
quality  field  work  was  caught  early  and  corrected  quickly.  Time  schedules 
were  maintained  due  to  the  use  of  deadline  dates  for  each  assignment.  In 
addition,  tight  control  facilitated  the  prompt  payment  of  interviewers,  an 
important  factor  in  interviewer  morale. 

Compensation  for  Field  Workers 
Supervisors'  Compensation 

Since  field  work  was  subcontracted  on  Study  Ml  and  because  there 
were  no  field  supervisors  on  Study  M4,  this  description  of  supervisors' 
compensation  will  be  related  only  to  Studies  M2  and  M3. 

Supervisors  were  paid  on  an  hourly  basis  plus  all  mileage  and  expenses. 
They  turned  in  a  weekly  time  and  expense  sheet  with  an  itemized  record 
of  all  miles  traveled  each  day,  phone  calls,  postage,  and  supplies.  They 
were  not  reimbursed  for  meals  or  other  personal  expenses. 

For  Study  M2,  the  beginning  hourly  rate  was  $1.85.  This  was  in- 
creased 5  cents  on  each  succeeding  wave,  so  that  by  Wave  5  the  super- 
visors were  earning  $2.05  per  hour.  In  addition,  they  were  compensated 
at  this  same  hourly  rate  for  any  interviewing  they  did  outside  of  their 
supervisory  functions. 

For  Study  M3,  a  flat  rate  of  $2.00  an  hour  was  paid  on  all  waves.  If 
the  supervisors  did  any  personal  interviewing  besides  their  supervisory 
work,  they  were  compensated  for  this  at  the  same  rate  per  interview  as 
the  other  interviewers. 

Interviewers'  Compensation 

Two  very  different  methods  of  compensating  interviewers  were  used. 
On  studies  Ml  and  M2,  interviewers  were  paid  on  an  hourly  basis  with 
expenses  for  mileage,  phone  calls,  and  miscellaneous  items.    On  Studies 
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M3  and  M4,  interviewers  were  paid  by  the  interview  supplemented  by 
hourly  pay  for  meetings. 

The  hourly  payment  method  was  adopted  at  the  beginning  of  the 
project  in  accordance  with  general  survey  practice  and  in  accordance  with 
the  dictates  of  much  of  the  literature  on  the  subject.  For  Study  Ml  the 
interviewers  were  paid  on  an  hourly  basis  by  the  interviewing  organiza- 
tions employed.  This  practice  was  continued  for  Study  M2.  It  is  gener- 
ally argued  that  payment  by  the  hour  encourages  interviewers  to  allow 
sufficient  time  to  conduct  the  interviews  and  not  to  feel  that  rushing  is 
necessary.  To  attract  good  candidates  the  rate  offered  was  somewhat 
above  the  going  hourly  rate  in  the  area.  Interviewers  were  offered  $1.65 
per  hour  for  all  time  spent  on  the  study,  including  meeting  and  study 
time.  In  addition,  interviewers  were  reimbursed  for  phone  calls,  postage, 
and  stationery,  and  received  5  cents  per  mile  for  mileage  (Figure  11). 
Furthermore,  as  an  incentive  to  remain  with  the  study,  the  hourly  rate 
was  raised  5  cents  on  each  wave  so  that  by  the  fifth  wave  the  inter- 
viewers were  earning  $1.85  per  hour. 

Experience  with  this  method  of  compensation  brought  out  various 
shortcomings.  For  one  thing,  high  variability  in  the  cost  per  interview 
was  apparent  from  one  interviewer  to  another.  In  part,  this  may  have 
been  due  to  padding  of  time  sheets  by  some  interviewers.  More  likely, 
however,  was  padding  of  a  different  sort;  namely,  stopping  at  a  sample 
address  if  it  were  near  the  place  the  interviewer  was  going,  even  though 
he  had  not  really  expected  to  find  the  sample  member  at  home. 

Another  problem  was  that  the  system  of  compensation  failed  to  pro- 
vide an  incentive  for  interviewers  to  complete  assignments  on  time.  Fur- 
thermore, the  5  cent  increase  per  wave  seemed  to  provide  virtually  no 
incentive  for  the  interviewers  to  remain  with  the  study.  Interviewers  who 
dropped  out  as  well  as  interviewers  who  remained  throughout  the  study 
felt  that  the  additional  pay  did  not  influence  their  decision.  Particularly 
pertinent  in  this  respect  were  the  comments  of  many  interviewers  that 
being  paid  on  an  hourly  basis  was  degrading  in  view  of  their  professional 
status  in  their  regular  jobs.  Their  feeling  seemed  to  be  that  payment  for 
work  accomplished  would  be  a  more  appropriate  system.  Moreover,  com- 
plaints were  registered  about  the  amount  of  paperwork  required  to  record 
hours  worked  and  expenses. 

After  considering  these  results,  it  was  decided  to  pay  the  interviewers 
on  Study  M3  by  the  completed  interview.  This  approach,  it  was  hoped, 
would  offset  the  objections  raised.  Moreover,  with  strict  control  over  the 
quality  of  the  interviewing,  it  was  felt  that  such  an  approach  would 
enable  interviewers  to  earn  a  more  satisfactory  hourly  rate  than  was 
possible  in  the  previous  studies  and  would  at  the  same  time  yield  more 
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reliable  data.  The  interviewers  would  not  have  to  spend  time  on  fringe 
or  unproductive  activities  such  as  filling  in  the  time  and  expense  record, 
or  making  two  or  three  trips  to  an  area  for  interviews  with  panel  members 
living  close  together  rather  than  trying  to  complete  them  all  on  the  same 
trip.  In  addition  it  was  hoped  that  they  would  be  more  efficient  in  trying 
to  make  contact  —  going  to  the  back  door  as  well  as  to  the  front  door 
in  trying  to  make  contact  at  a  sample  address.  In  short,  interviewer 
activities  would  be  directed  toward  the  completion  of  an  interview  and 
the  obtaining  of  accurate  and  complete  information. 

The  average  rates  per  interview  paid  by  the  hour  for  Studies  Ml  and 
M2  were  used  as  the  basis  for  setting  the  rates  per  interview  for  Study  M3. 
The  rates  established  were  $10  for  each  of  the  first  10  completed  inter- 
views, $13  each  for  all  interviews  over  10,  and  $15  for  each  interview 
obtained  from  a  panel  member  who  had  previously  refused  another  inter- 
viewer. Once  the  $10  or  $13  rate  was  established,  it  remained  constant 
for  the  remaining  four  waves,  regardless  of  the  interviewer.  If  an  inter- 
viewer persuaded  a  panel  member  who  had  previously  refused  an  inter- 
view to  grant  one,  the  rate  for  this  respondent  moved  into  the  $15 
bracket,  where  it  remained.  It  was  possible  for  this  to  happen  on  any 
wave. 

These  arrangements  were  made  clear  to  the  interviewers  at  the  train- 
ing sessions  and  were  contained  in  the  "Interviewer  Agreement"  (Appen- 
dix B),  which  each  interviewer  was  required  to  read  and  sign.  Inter- 
viewers were  also  told  that  payment  would  be  made  only  after  interviews 
had  been  completed  and  the  questionnaires  had  been  edited.  If  a  ques- 
tionnaire was  not  completed  or  if  it  contained  many  blanks,  it  was  sent 
back  for  further  work,  and  the  payment  of  the  bill  for  that  interview  was 
deferred. 

From  Table  8  it  is  apparent  that  the  rates  established  for  Study  M3 
were  well  below  the  average  of  Studies  Ml  and  M2.  However,  the  earn- 
ings of  some  of  the  more  effective  interviewers  in  Studies  Ml  and  M2 
were  consistently  below  the  rates  established  for  Study  M3.  As  a  result,  it 
was  felt  that  the  $10  and  $13  rates  were  realistic.  On  Wave  1  of  Study 
M3,  the  interviewers'  reactions  to  these  rates  were  closely  watched.  No 
apparent  loss  in  morale  or  drop  in  quality  of  interviewing  was  observed. 
If  the  rates  had  proved  to  be  detrimental,  they  could  have  been  raised  to 
the  necessary  levels.  However,  no  such  problems  occurred.  In  fact,  the 
interviewers  seemed  enthusiastic  about  how  much  they  were  earning. 
Much  of  the  wasted  time  and  motion  had  been  eliminated  whereas  high 
response  and  pick-up  rates  were  being  maintained. 

These  rates  covered  all  time  and  expenses  required  to  obtain  an  inter- 
view, with  the  exception  that  interviewers  were  reimbursed  at  the  rate  of 
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Table  8.  Average  Cost  per  Interview  on  All  Waves  of  All  Studies 

(Dollars) 


Wave 

Study 
Ml 

Study 
M2a 

Study 
M3 

Study 

M4 

1 

2 

3 

4 

5 

18. 871' 

18.46 

18.23 

18.19 

19.02 

23.11 
20.61 
15.42 
14.41 
10.03 

16.87 
13.62 
13.23 
12.58 
13.68 

14.79 
13.15 
14.35 
12.49 
13.92 

18.55 

16.72 

14.00 

13  74 

a  The  large  decrease  in  the  cost  per  interview  of  $13.08  from  Wave  1  to  Wave  5  was  due 
mainly  to  the  improvement  of  field  procedures  and  control.  This  was  the  first  study  where  the 
field  was  under  the  direct  supervision  of  project  personnel. 

b  This  cost  figure  is  relatively  low  because  this  panel  operation  had  been  preceded  by  a 
so-called  ''background  interview"  with  the  same  panel  members  about  six  months  earlier,  so  that 
much  of  the  original  contact  work  was  out  of  the  way  by  the  first  wave. 


8  cents  per  mile  for  panel  members  living  more  than  10  miles  from 
their  homes;  the  rate  applied  only  to  mileage  over  10  in  any  direction. 
Interviewers  were  also  paid  for  training  and  study  time  at  the  rate  of 
$1.50  per  hour. 

The  written  instructions  stated  that  bills  would  be  processed  every 
three  weeks.  On  those  dates,  all  bills  being  held  were  sent  to  the  Univer- 
sity Business  Office  for  payment.  Interviewers  were  told  to  call  the  office 
if  they  had  a  check  coming  which  had  not  arrived  in  a  week  or  10  days 
after  the  payment  dates. 

The  method  of  payment  by  interview  was  also  used  on  Study  M4, 
with  two  modifications.  During  each  wave  the  rate  per  interview  was  $10 
for  each  of  the  first  10  interviews  and  $13  for  each  interview  over  10. 
Once  the  $15  rate  was  established  for  reluctant  panel  members,  it  re- 
mained constant  on  all  succeeding  waves.  The  $13  rate  after  10  inter- 
views had  the  effect  of  speeding  up  the  field  operation,  since  interviewers 
were  encouraged  to  reach  the  higher  rate.  The  assigning  of  additional 
interviews,  however,  was  at  the  discretion  of  the  field  director.  There- 
fore, if  an  interviewer  was  doing  work  that  was  hurried  and  of  question- 
able quality,  he  was  not  given  more  assignments. 

A  second  change  was  that  Study  M4  interviewers  were  paid  for  excess 
mileage  (over  10  miles)  on  all  assignments  worked,  regardless  of  whether 
the  outcome  was  an  interview,  a  refusal,  or  noncontact.  The  purpose  of 
this  was  to  cover  some  of  the  interviewer's  expenses  incurred  in  attempts 
to  contact  panel  members  living  far  from  the  interviewer's  home.  Reim- 
bursement for  excess  mileage  on  this  basis  seemed  proper  because  farm 
respondents  might  be  spread  over  a  wide  area. 

All   things  considered,   the   procedure   of  paying  by   the   completed 
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interview  seemed  to  be  much  superior  to  that  of  paying  by  the  hour.40  In 
large  measure,  this  was  undoubtedly  due  to  the  professional  background 
of  most  of  the  interviewers  and  the  repeated  emphasis  placed  on  the  nec- 
essity of  obtaining  complete  and  accurate  information.  As  a  result,  there 
was  little  evidence  that  interviewers  were  rushing  respondents  to  shorten 
the  interview.  At  the  same  time,  there  was  some  incentive  for  inter- 
viewers not  to  prolong  the  interviews  if  they  were  not  obtaining  relevant 
information,  because  they  were  not  then  receiving  any  additional  pay. 

In  terms  of  cost  per  interview,  compensation  by  the  interview  was 
more  economical  than  compensation  by  the  hour,  as  is  shown  by  the 
schedule  of  average  cost  per  interview  for  each  wave  on  each  of  the  four 
studies  in  Table  8. 

These  data  cover  all  costs  connected  with  the  interviews,  including 
compensation  for  meetings,  mileage  expenses,  and  trips  made  by  project 
staff  members  to  the  field.  As  is  evident  from  this  tabulation,  the  average 
cost  per  interview  in  the  last  two  studies  was  appreciably  below  that  in 
the  first  two.  The  cost  shown  for  Study  Ml  is  not  directly  comparable 
with  the  costs  shown  for  the  other  studies  because  it  was  subcontracted. 
Nevertheless,  this  tabulation  does  suggest  that  payment  by  the  interview, 
even  at  very  respectable  rates,  is  likely  to  be  more  economical  than  pay- 
ment by  the  hour. 

Recommendations  for  the  Future 

Continuous  contact  would  appear  to  be  the  key  to  adequate  inter- 
viewer supervision.  If  feasible,  communication  should  be  handled  by  a 
field  director  who  is  thoroughly  conversant  with  the  purpose  and  objec- 
tives of  the  study  and  with  the  types  of  data  that  are  being  sought.  In 
large-scale  studies  this  is  not  possible  and  field  supervisors  must  be  used. 
In  such  cases,  the  principle  of  continuous  contact  should  still  be  main- 
tained with  the  lines  of  communication  going  from  the  field  director  to 
the  supervisors,  and  from  the  supervisors  to  the  interviewers.  •  This  also 
indicates  the  need  for  extensive  and  intensive  training  and  involvement 
of  supervisors  if  they  are  to  perform  the  functions  of  the  field  directors 
for  smaller  studies. 

Particularly  important  to  the  maintenance  of  the  quality  of  data  de- 
sired are  strict  controls  at  the  very  beginning  of  an  operation.  During 
the  training  sessions  interviewers  should  be  told  of  the  planned  use  of  such 
controls,  and  these  controls  should  be  put  into  effect  as  soon  as  the  field 
operation  begins.    If  interviewers  are  given  to  understand  at  the  very 

40  For  a  thorough  analysis  and  comparison  of  these  two  methods  of  interviewer 
compensation,  see  Mathew  Hauck,  "Interviewer  Compensation  on  Consumer  Sur- 
veys." Commentary,  The  Journal  of  the  Market  Research  Society,  No.  14  (Sum- 
mer, 1964),  pp.  15-18. 
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beginning  that  incomplete  or  careless  work  will  not  be  accepted  and  that 
they  will  be  remunerated  only  if  the  interviews  do  meet  certain  standards, 
and  if  they  see  that  these  principles  are  enforced  at  the  outset,  they  will 
be  strongly  motivated  to  carry  out  their  assignments  in  a  thorough  manner. 

In  addition,  with  the  maintenance  of  strict  controls  on  the  quality  of 
interviewing,  payment  by  the  interview  seems  to  be  a  much  more  efficient 
approach  than  payment  by  the  hour.  This  is  particularly  true  if  profes- 
sional people  are  hired  as  interviewers,  for  this  method  of  compensation 
serves  to  place  extra  responsibility  on  the  interviewers,  while  at  the  same 
time  offering  more  rewards. 

An  important  function  of  proper  supervision  is  to  avoid  overloading 
interviewers  with  assignments.  It  would  seem  best  to  give  each  interviewer 
only  a  few  assignments  at  a  time,  with  a  deadline  date  for  these  assign- 
ments. After  all  of  these  assignments  have  been  turned  in,  a  new  group  of 
assignments  may  be  sent  out  together  with  a  new  deadline  date.  In  this 
manner,  tight  control  can  be  maintained  over  the  flow  of  interviews. 
Also,  this  procedure  serves  to  counteract  the  tendency  of  many  inter- 
viewers to  put  off  contacting  sample  members  whom  they  feel  may  be 
difficult  to  interview.  It  is  not  unlikely  that  at  times  this  very  habit  of 
tardiness  makes  a  sample  member  less  cooperative,  especially  when  an 
advance  letter  had  been  received  many  days  or  weeks  earlier. 

If  resources  permit,  a  refresher  training  session  after  the  first  few  days 
of  interviewing,  or  after  the  first  five  or  six  interviews,  is  a  very  useful 
device  for  discovering  common  problems  and  for  clearing  up  questions  or 
difficulties  that  the  interviewers  may  have  encountered  in  the  field.  Such 
a  meeting  can  also  prove  very  useful  for  promoting  data  standardization. 

The  practice  of  having  the  interviewers  edit  their  own  questionnaires, 
although  it  is  no  substitute  for  office  editing,  appears  to  be  of  real  value 
in  reducing  omissions  and  "don't  know"  answers.  This  is  particularly  true 
if  the  interviewers  know  that  unsatisfactory  questionnaires  will  be  re- 
turned to  them  for  additional  (noncompensated)  work. 

Spot-check  post  cards  and  the  switching  of  interviewers'  assignments 
can  be  useful  in  reducing  interviewer  cheating,  particularly  when  inter- 
viewers are  told  at  the  initial  training  sessions  of  their  planned  use. 

Finally,  a  supervisor  must  evaluate  the  effectiveness  of  each  inter- 
viewer. When  interviewers  are  not  meeting  project  standards  of  complete- 
ness and  accuracy,  they  can  be  dismissed  or  retrained. 

Thus  many  different  aspects  of  interviewing  performance  must  be 
used  to  obtain  a  reliable  indicator  of  the  completeness  and  accuracy  of  the 
data.  The  results  presented  in  this  chapter  suggest  that  it  is  possible  to 
develop  better  means  of  identifying  effective  interviewers  on  the  basis  of 
their  performance. 
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APPENDIX  A 

INTERVIEWER  QUALIFYING  EXAMINATION 
Study  of  Farm  Family  Finances 

1.  In  your  own  words,  please  explain  the  objective(s)  of  this  study,  i.e.,  what 
we  hope  to  gain  from  it. 


2,     Suppose  a  farmer  does  not  believe  it  is  necessary  for  him  to  be  interviewed. 
Which  of  the  following  explanations  would  be  appropriate? 

a.  The  selection  was  done  in  a  purely  random  manner. 


b.  The  refusal  of  even  one  respondent  lowers  the  amount 
of  information  we  have  available. 

c.  We  cannot  substitute  names. 

d.  None  of  the  above. 


3.  Currently  there  are  several  reliable  sources  of  family  savings  data,  and 
this  project  aims  to  develop  another  source  to  be  used  as  a  check.  T   F 

h.     In  each  of  the  following  situations,  briefly  explain  how  you  would  overcome 
the  objections. 

a.  A  small  acreage  farmer  cannot  understand  why  his  name  was  chosen  and  not 
the  people  down  the  road. 


b.  A  wealthy  farmer  cannot  understand  why  his  name  was  chosen.  Besides,  he 
is  too  busy. 
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A  sample  member  states  he  does  not  have  much  savings  and  therefore  does 
not  have  to  be  interviewed.     


d.  The  farm  operator  and  his  wife  are  both  at  home.  They  refuse  to  be  in- 
terviewed because  there  have  been  several  "surveyors"  around  recently  whc 
have  turned  out  to  be  salesmen. 


The  farm  house  is  run  down  and  poor  looking.  The  farmer  refuses  to  be 
interviewed  because  he  never  did  like  universities  and  refuses  to  have 
anything  to  do  with  them. 


A  middle-aged  farmer  refuses  because  he  believes  that  the  interviewer  is 
just  checking  on  his  income  tax  report. 


A  sample  member,  while  being  interviewed,  refuses  to  go  any  further 
because  the  questions  are  too  personal. 
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h.  A  harried-looking  housewife  says,  "I'm  busy. 


5.  Which,  if  any,  of  the  following  is  the  recommended  approach  in  contacting 
the  farmer? 

a.  Go  to  the  respondent's  house  and  attempt  either  to  get  an 

immediate  interview  or  a  firm  appointment. 


b.  Telephone  the  respondent  and  make  an  appointment  to  see  him. 

c.  Go  to  the  respondent's  home,  fill  in  the  classification  data 
and  make  an  appointment  to  go  back  when  all  the  adult  family 
members  can  be  present. 

d.  Go  to  the  respondent's  house  and  interview  them  if  the  entire 
adult  family  is  present;  otherwise,  make  an  appointment  to 
return  when  they  are  all  present. 

e.  None  of  the  above. 


6.  (True  or  False)  The  Inter-University  Committee  for  Research  on  Consumer 
Behavior  is  a  branch  of  the  Ford  Foundation  and  was  organized  to  conduct 
this  study.   T   F 

7.  In  your  own  words  explain  why  the  farm  study  is  being  done. 


During  the  interview  the  respondent  digresses  and  goes  into  a  long  descrip- 
tion of  last  year's  poor  production.  You  should: 

a.  Change  the  subject  quickly  back  to  the  interview. 


Say,  "Yes,  that's  very  interesting,  but  I  know  your  time  is 
precious  so  to  get  back  to  the  interview..." 

Listen  attentively  and  then  ask  whatever  financial  questions 
may  be  pertinent  to  the  subject. 

None  of  the  above. 
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9.  Briefly  explain  why  records  are  important  and  why  every  attempt  should  be 
made  to  get  the  farmer  to  refer  to  them  at  all  times. 


10.  If  a  respondent  tries  to  avoid  being  interviewed,  which  of  the  following,  if 
any,  is  (are)  the  wrong  method(s)  of  proceeding? 

a.  Put  on  a  "hard  sell." 

b.  Say  you  will  call  again  in  two  weeks. 

c.  Make  it  clear  that  you  are  willing  to  come  at  any  time  that 

is  convenient. 

d.  Always  try  to  forestall  an  outright  refusal. 

e.  None  of  the  above. 


11.  Two  different  experiments  are  being  used  on  the  first  wave.  List  three 

experiments  and  briefly  tell  what  you  must  know  before  attempting  to  conduct 
an  interview.  


12.  What  would  you  say  to  a  farmer  if  he  said  this  is  his  busy  time  and  he  has 
no  time  to  stop  and  talk? 


13.  If  a  farmer  does  not  comprehend  the  importance  of  the  study,  which  of  the 
following,  if  any,  is  (are)  the  correct  way(s)  to  continue? 

a.  Explain  the  fact  that  this  study  will  bring  to  light 

important  figures  on  the  savings  of  the  farm  population. 


b.  Restate  the  fact  that  their  name  will  not  be  used. 
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c.  Stress  the  importance  of  each  farmer  in  the  sample  and  how  he 
cannot  he  replaced. 

d«  Ignore  the  questions  of  the  farmer  and  go  on  with  the  inter- 
view. 

e.  None  of  the  ahove. 


Ik.   The  respondent  says  he  doesn't  know  how  much  money  he  has  in  his  checking 
account,  and  it  develops  that  he  doesn't  know  offhand,  hut  his  checkhook 
is  lying  on  top  of  the  dresser  upstairs.  You  should: 

a.  Ask  him  to  give  you  an  estimate. 

h.  Tell  him  that  you  will  call  hack  later  for  the  information. 


c.  Chide  him  for  not  heing  more  cooperative,  and  try  to  shame 
him  into  looking  at  his  records. 

d.  Re-emphasize  that  the  study  is  of  nationwide  importance,  and 
that  we  are  seeking  complete  and  accurate  information. 

e.  None  of  the  ahove. 


15.  (True  or  False)  Under  no  circumstances  should  the  panel  nature  of  the  study 
"be  mentioned  until  the  interview  is  completed.  T   F 

16.  Within  a  given  savings  unit  we  want  to  interview: 
a.  The  oldest  person  in  the  savings  unit. 

h.  The  person  in  the  savings  unit  who  has  the  most  money. 

c.  The  person  who  makes  the  decisions  on  saving  and  horrowing. 

d.  The  person  who  keeps  the  records  on  spending  and  saving. 

e.  The  person  who  spends  the  most  money. 

f .  None  of  the  ahove. 

17.  In  connection  with  Card  D  (Dollar  Brackets  for  Broad  Asset  Holdings),  we  are 
interested  in  ohtaining: 

a.  Only  the  holdings  of  the  farm  operator. 

h.  An  exact  dollar  amount  for  each  holding  from  his  records. 


c.  Only  in  the  holdings  that  the  savings  unit  has.  Leave  other 
holdings  hlank. 

d.  None  of  the  ahove. 
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18.  The  farm  operator  is  married  and  has  3  daughters,  ages  18,  7  and  h.     He  in- 
dicates that  he  has  only  the  farm,  a  checking  account,  and  some  government 
bonds,  and  his  wife  owns  some  stock.  What  is  the  next  thing  you  would 
probe  for? 


19.  (True  or  False)  After  you  find  out  with  Card  D  what  assets  the  savings 
unit  owns,  you  will  not  have  to  ask  about  the  ones  they  do  not  own  when  you 
get  to  detailed  asset  holdings.  T   F 

20.  (True  or  False)  In  conducting  an  interview,  always  cover  all  questions  in 
the  order  in  which  they  are  listed.  T   F 

The  following  statements  21-^8  are  definitions  of  some  of  the  terms  that  you 
should  be  familiar  with  for  this  study.  If  the  statement  is  True  put  a  T  in  the 
space  provided.  If  the  statement  is  False  place  an  F  in  the  space  provided  and 
correct  the  statement.  The  question  will  be  counted  wrong  if  it  is  marked  False 
and  not  corrected  or  if  the  correction  is  in  error. 

21.  A  mortgage  bond  is  held  by  an  individual  or  a  concern. 


22.  Dividends  paid  on  common  stock  are  not  fixed  and  usually  depend  on  the 
profits  of  the  firm  during  the  previous  year. 

23.  For   recording  purposes,  we  want  to  separate  cash  from  stock  holdings  in 
a  brokerage  account. 

2k.     Preferred  stock  pay  varying  dividends  just  like  common  stock.  It  is 

"preferred"  because  its  holder  has  prior  claim  against  the  assets  of  the 
firm  in  case  of  liquidation. 

25.  Public  authority  bonds  are  issued  by  a  federal  authority. 


26.  A  contract  differs  from  a  mortgage  in  that  the  purchaser  receives  title  to 
the  real  estate  after  a  specified  amount  is  paid  off. 

27.  Any  interest  accruing  to  a  Postal  Savings  account  is  added  to  that  account 
and  can  be  withdrawn  when  the  certificate  is  cashed. 

28.  A  dwelling  unit  is  a  group  of  rooms  with  kitchen  facilities.  Thus,  a 
dwelling  unit  may  be  a  house,  apartment,  trailer,  etc. 

29.  Mortgage  insurance  is  a  form  of  endowment  insurance. 


30.  An  annuity  contract  provides  for  the  payment  of  a  sum  of  money  in  return  for 
which  an  individual  is  guaranteed  this  sum  of  money  plus  a  stipulated 
interest  in  a  lump  sum  at  some  future  period. 

31.  Endowment  insurance  provides  a  definite  sum  of  money  only  to  the  policy 
holder's  beneficiary  after  his  death. 
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32.  Under  straight  life  insurance,  premiums  may  be  payable  until  death,  or 
premiums  may  be  payable  for  a  specified  number  of  years . 

33.  The  farm  operator  is  the  person  earning  the  largest  income  in  the  savings 
unit.  In  case  of  equal  earnings,  it  is  the  older  male  in  the  savings 
unit. 

3*+.  The  denomination  of  United  States  Government  Bonds  is  the  amount  re- 
ceivable when  the  bond  matures. 

35.  Series  E  Bonds  may  be  cashed  in  only  after  10  years  if  you  want  to  get  any 
interest.  If  the  bond  is  cashed  before  10  years,  the  holder  only  gets  back 
what  he  paid  originally. 

36.  Savings  and  loan  associations  have  two  kinds  of  savings  accounts.  They  are 
(l)  savings  share  accounts,  where  the  interest  is  mailed  to  the  owner  of 
the  account,  and  (2)  investment  share  accounts,  where  the  interest  is  added 
to  the  account. 

37.  Group  life  insurance  is  a  form  of  term  life  insurance.  The  members  of  the 
group  usually  hold  certificates  as  evidence  of  their  insurance. 

38.  Interest  on  Series  H  Bonds,  United  States  Treasury  Bills,  and  United  States 
Treasury  Notes  is  sent  periodically  to  the  holder. 

39-  If  "the  holder  of  a  savings  life  insurance  policy  dies,  his  beneficiaries 
receive  an  amount  equal  to  the  balance  in  the  account. 

*+0.  A  savings  unit  consists  of  one  or  more  related  persons  living  in  the  same 
dwelling  unit  who  pool  half  or  more  of  their  income  and  savings. 


hi.     A  mutual  fund  sells  its  stock  or  shares  to  investors  and  then  invests  the 
proceeds  in  stocks  and  bonds  of  other  corporations.  Dividends  are  paid 
directly  from  these  corporations  to  the  mutual  fund  holders  just  as  they 
pay  stockholders  direct. 

*+2.  An  investment  club  is  a  club  to  which  a  group  of  people  belong  into  which 
each  member  puts  a  certain  sum  of  money  per  week,  month,  etc.  The  club 
splits  profits  and  dividends. 

J+3-  The  face  value  of  a  life  insurance  policy  is  the  amount  stated  on  the  front 
of  each  policy  that  will  be  paid  in  case  of  death  or  at  maturity  of 
policy . 

kk.     A  paid-up  life  policy  is  one  for  which  all  the  premiums  have  been  paid  and 
therefore  the  policy  is  no  longer  in  force . 

45.  A  checking  account  may  be  held  in  either  a  bank  or  a  savings  and  loan 
association. 

U6.     Debenture  bonds  are  secured  by  specific  assets  of  the  firm. 


kj.     If  a  savings  unit  owes  money  to  a  credit  union,  it  should  be  recorded  as 
an  asset. 
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48.  If  a  farm  operator  does  not  know  the  value  of  his  machinery,  leave  the 
question  blank. 

h$,     -to  your  own  words,  how  would  you  explain  to  a  farmer  how  he  happened  to  he 
picked  for  this  study? 


50.  (True  or  False)  For  savings  accounts  (holdings  and  change  forms)  we  want 
to  know,  among  other  things,  the  date  of  the  last  deposit  and  the  date  of 
the  last  withdrawal.   T  F 

51.  (True  or  False)  Expected  total  income  for  i960  is  that  of  the  farm 
operator  only.   T  F 

52.  (True  or  False)  We  want  to  know  the  name  of  a  mortgage  holder  only  if  the 
mortgage  is  cured  to  an  institution.  If  a  mortgage  is  held  by  an  individual 
the  name  is  not  necessary — merely  "mother-in-law,"  "uncle,"  etc.,  will  be 
sufficient .   T  F 

53.  (True  or  False)  Carrying  charges  are  not  to  be  included  in  the  present 
amount  owed  on  a  debt.   T  F 

54.  In  order  to  keep  track  of  items  you  may  want  to  recall  for  the  next  inter- 
view, you  should: 

. a.  Write  them  under  "interviewer's  Notes"  on  the  IRF. 


b.  Not  write  them  under  "interviewer's  Notes,"  because  the 

information  in  this  space  is  to  be  tabulated. 

c.  Not  write  them  under  "interviewer's  Notes,"  because  the 

IEF  will  not  be  returned  to  you  on  the  next  wave. 

d.  Keep  a  separate  file  of  notes  on  your  panel  members. 

55.  (True  or  False)  A  member  of  the  family  who  is  either  looking  for  work  or 
has  been  temporarily  laid  off  is  considered  employed.   T  F 

56.  The  respondent  has  a  savings  account,  but  knows  of  another  institution 
where  he  can  get  a  higher  rate.  He  plans  to  open  up  a  new  account 
there.  How  would  you  record  this? 


57.   (True  or  False)  The  total  investment  in  an  annuity  is  the  amount  of  the 
down  payment.   T  F 
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58.  (True  or  False)  Only  interview  sample  members  who  are  presently  engaged  in 
farming .  T   F 

59.  In  your  own  words,  describe  the  step-by-step  procedure  from  the  time  you 
receive  your  assignments  until  the  completed  interview  is  mailed  in: 


60.  Explain  briefly  the  meaning  of  a  personal  trust. 


61.  If  a  stock  does  not  sound  as  if  it  is  on  a  "Listed  Exchange,"  you  should: 
a.  Look  it  up  in  the  paper. 

______  b.  Not  record  it. 

c.  Get  the  price  from  the  respondent. 

d.  Write  "not  listed"  instead  of  the  price  per  share. 

e.  None  of  the  above. 

62.  If  you  find  out  that  &   family  is  away  on  vacation  for  the  next  couple  of 
months,  what  parts  of  the  Interviewer  Report  Form  do  you  fill  out? 


63.  (True  or  False)  The  dollar  amount  the  saving  unit  expects  to  receive  in 
i960  from  all  sources  of  income  is  to  be  totaled.   T   F 
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6k.     The  farm  operator  owns  five  Series  E  Bonds,  two  of  which  axe  $50  denomina- 
tions, one  of  which  is  $100,  and  two  of  which  are  $25;  he  also  owns  three 
$100  Treasury  Certificates.  Show  how  you  would  list  these  "bonds. 


65.  (True  or  False)  With  regard  to  insurance  premiums,  we  are  interested  in 
what  is  paid  and  how  often.  T   F 

66.  A  checking  account  had  a  balance  of  $1538.25  on  July  1,  i960,  and  at  the 
time  of  the  interview  has  a  balance  $1^23.85.  This  would  mean  the  balance 
on  the  holding  form  is: 

a.  Up  (U)  by  $lll+.»40. 

b.  The  same  (s). 

c.  Down  (D)  by  $11^.^0. 

d.  None  of  the  above. 


67.  If  a  family  refuses  to  be  interviewed,  what  parts  of  the  IRF  do  you  fill 
out? 


68.  What  does  "Atyr"  mean,  and  why  is  it  used? 


69.  Explain  the  differences  between  accuracy  and  completeness. 


70.  (True  or  False)  The  Classification  Data  (Part  2  of  the  IRF)  can  be  left 
blank  if  an  interview  is  obtained  because  all  of  the  requested  information 
will  be  recorded  on  the  other  forms.  T   F 

71.  (True  or  False)  Someone  in  the  savings  unit  has  two  insurance  policies, 
one  with  Mutual  of  Omaha  and  the  other  with  Mutual  of  New  York.  It  will 
be  sufficient  for  our  purposes  if  you  record  them  both  as  "Mutual."  T   F 
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72.  Suppose  you  have  to  visit  a  family  three  times  before  you  obtain  an  inter- 
view. The  first  time,  August  23  at  7:00  p.m.,  no  one  is  at  home.  The 
second  time,  August  25  at  U:00  p.m.,  the  farm  operator  says  he  is  too  busy 
to  be  interviewed  and  you  make  an  appointment  for  August  27  at  10:00  a.m. 
The  third  time,  August  27  at  10:00  a.m.,  you  obtain  the  interview  from  the 
farm  operator  and  his  wife.  How  do  you  record  this  information  under 
"Contact  Report"  in  the  IRF? 

Approx. 
Date   Day   Time    Where   Persons(s)  talked  to    age    Results  of  Contact 


73«  Briefly  tell  what  we  want  to  know  about  life  insurance  on  the  C  Form  with 

regard  to:  Where  did  the  money  come  from?  or  What  was  done  with  the  money? 


7h.     The  respondent  has  bought  a  new  insurance  policy  since  July  1.  Among  other 

things  on  the  change  form,  we  want  to  know: 

a.  Amount  received. 

b.  Amount  that  has  been  piad  on  the  policy  since  July  1. 

c.  Amount  the  policy  is  worth  with  a  double  indemnity  clause. 

d.  Face  value  of  the  policy. 

e.  None  of  the  above. 


75  •  In  the  change  approach  if  a  checking  account  has  been  opened  since  July  1 
we  want  to  know  the  source  of  the  money.  If  a  checking  account  has  been 
closed  since  July  1,  we  want  to  know  the  use  of  the  money.  If  there  has 
been  a  change  in  balance  we  want  to  know  the  source.   T   F 

76.  Consider  the  following  questions  as  referring  to  the  following  family 

situation.  The  farm  operator  is  a  man  in  his  late  forties.  He  operates  210 
acres.  He  has  a /wife  and  three  children:  daughter  (19)  a  college  junior, 
son  (17)  who  is  just  finishing  high  school  and  helps  the  operator  on  the 
farm,  and  daughter  (l1*)  who  just  finished  her  first  year  of  high  school. 
The  family  lives  in  a  neatly  kept  nine-room  farm  house. 

a.  On  broad  asset  holdings,  the  farm  operator  replies  that  he  and  his 
wife  have  a  checking  account.  When  you  are  obtaining  detailed  infor- 
mation, the  farm  operator  gives  you  the  data  on  the  account.  Before 
going  on,  and  in  view  of  the  family  composition,  what  should  you  query 
about? 
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h.  The  farm  operator  replies  that  he  has  no  savings  accounts  and  has  all 
his  investment  tied  up  in  the  farm.   In  view  of  the  family  composition., 
what  would  he  the  next  question? 


c.  List  four  types  of  life  insurance  that  the  family  might  own. 


d.  The  wife  says  that  she  has  some  life  insurance  that  her  parents  had 

taken  out  for  her  hut  which  she  had  all  hut  forgotten.  She  is  not  sure 
that  it  is  still  in  effect.  She  says  that  it  is  just  a  nominal  amount 
and  is  very  vague  ahout  it  all.  Briefly,  what  would  you  do? 


The  wife  states  that  she  is  part  owner  (with  her  mother)  of  some  savings 
honds.  She  knows  the  number  and  denomination.  Briefly,  what  would  you 
do? 


f.  If  only  the  wife  is  mentioned  as  owning  savings  honds,  what  would  this 
prompt  you  to  do? 

g.  The  farm  operator  has  to  leave  the  interview  situation  and  the  wife 
says  she  will  answer  the  questions.  After  a  few  questions,  it  appears 
she  does  not  have  the  answers.  What  would  you  do? 


Date Name 

Address 
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APPENDIX  B 

INTERVIEWER  AGREEMENT 
Study  of  Farm  Family  Finances 

Background  and  purpose 

The  study  in  which  you  have  been  engaged  as  an  interviewer  has  as  its  objective 
the  collection  of  accurate,  up-to-date  information  on  farm  savings  practices  and 
attitudes.  You  have  undertaken  to  cooperate  in  the  study  and  assist  in  the 
performance  of  a  very  necessary  part  of  it  by  utilizing  your  particular  skills 
in  an  area  of  interviewing  which  demands  independence  of  control  and  an  objec- 
tivity at  the  interview  level.  The  success  of  this  study  will  be  dependent  upon 
the  performance  of  those  skills  and  their  utilization  by  you;  therefore,  you 
will  be  given  a  great  measure  of  freedom  in  the  selection  of  time  and  method. 
Tne  major. concern  of  the  study  will  be  the  results  achieved  by  you,  the  inter- 
viewer . 

General  arrangements 

You  will  have  submitted  to  you  the  names  and  addresses  of  approximately  fifteen 
farmers  to  be  interviewed  in  your  interview  area.  An  attempt  will  be  made  to 
restrict  interviews  to  a  10-mile  radius  of  your  home.  If  a  sample  member  re- 
sides outside  the  interview  area,  you  will  be  reimbursed  at  the  rate  of  5  cents 
per  mile  for  every  mile  beyond  the  10-mile  radius  each  way  necessary  to  contact 
the  sample  member. 

You  will  be  paid  on  a  contract  basis  by  the  completed  interview.  The  pay 
schedule  for  completed  interviews  is  as  follows: 

a.  For  each  of  the  first  ten  completed  interviews  on  each  wave,  $10.00 
per  interview. 

b.  For  all  completed  interviews  over  ten  on  each  wave,  $13.00  per 
interview. 

c.  For  interviews  on  which  a  previous  interviewer  has  received  a  refusal, 
$15.00  per  interview.  In  addition,  $15.00  will  be  paid  for  interviews 
with  this  panel  member  on  each  successive  wave. 

As  an  interviewer  for  this  study,  you  are  functioning  as  an  independent  contractor, 
not  as  an  employee  of  the  University  of  Illinois.  Contract  payments  will  be 
made  to  you  on  receipt  of  bills  without  any  withholding  for  income  tax,  retire- 
ment, insurance,  Social  Security,  or  any  like  or  dissimilar  deductions. 

It  may  be  necessary  from  time  to  time  to  meet  to  discuss  the  review  procedures 
and  developments  relating  to  the  study.   Ordinarily  these  meetings  will  take 
the  form  of  a  general  briefing  session  at  the  outset  of  each  wave  of  interviews, 
a  review  meeting  at  about  the  middle  of  each  wave,  and  a  final  meeting  at  the 
end  of  each  wave  of  interviews.  The  meetings  will  be  set  at  a  time  and  place 
most  convenient  to  all  interviewers.  You  will  receive  the  sum  of  $1.65  per  hour 
for  attending  these  meetings.  No  travel  time  or  mileage  payments  will  be  made 
for  attending  these  meetings . 
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Interviewing 

To  give  substance  to  this  study  and  insure  that  your  skills  are  advantageously 
utilized,  here  are  a  few  general  guides: 

a.  Interviewers  are  expected  to  complete  about  12  interviews  on  each  wave. 
In  addition,  interviewers  are  expected  to  attempt  to  make  contact  at  the 
designated  address  within  four  days  after  receiving  the  names  and 
addresses  of  the  interviewees. 

b.  To  produce  the  best  results,  all  interviews  should  be  conducted  in  the 
home  of  the  sample  member,  or  any  other  place  conducive  to  privacy  and 
convenient  to  the  sample  member.   Interviews  should  not  be  conducted  in 
a  public  place  nor  while  neighbors  and/ or  relatives  are  present. 

c.  A  completed  interview  should  cover  all  questions  sought  by  this  study 
and  should  contain  no  unexplained  blank  spaces. 

d.  Completed  interviews  should  be  transmitted  by  mail  no  later  than  9 
o'clock  in  the  morning  of  the  day  following  that  in  which  they  have 
been  completed. 

e.  Information  obtained  from  a  sample  member  must  not  be  discussed  with 
anyone  other  than  project  personnnel. 

f .  It  is  essential  that  interviewers  be  available  to  interview  on  a  part- 
time  basis,  principally  evenings  and  weekends,  for  a  period  of  one 
year. 


I  have  read  and  understand  the  foregoing  material  included  in  this  Interviewer 
Agreement . 

(signed) 

Date 
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ATTITUDE  OF  PANEL  MEMBER 

a.  In  your  opinion,  how  accurate  and  how  complete  is  the  information  that 
was  given  for  each  of  the  following  holdings. 


y   Fully  so.... 

g   Fairly  so . . . 

<      Not  much.. .. 

Very  little. 


Checking 

Savings 

Life 

Govern- 

Own 

accounts 

accounts 

insur- 

ment 

busi- 

ance 

bonds 
□ 

ness 
□ 

Stock 

D 

Debts 

D 

D 

D 

□ 

D 

D 

D 

□ 

D 

D 

a 

D 

D 

D 

□ 

□ 

D 

a 

D 

□ 

D 

D 

D 

□ 

a 

D 

□ 

□ 

□ 

D 

D 

□ 

□ 

D 

D 

□ 

D 

□ 

□ 

D 

□ 

D 

D 

D 

□ 

D 

D 

D 

D 

D 

□ 

D 

D 

Fully  so.... 
Fairly  so... 
Not  much. ... 
Very  little. 


Not  applicable. 


□ 


□ 


D 


□ 


□ 


D 


D 


b.  What  was  respondent's  attitude  toward  the  use  of  records  in  supplying  the 
figures  requested? 


Did  you  have  the  impression  at  any  time  that  the  respondent  did  not  know 
the  figures  too  well  which  he  gave  you?  nYes  Q  No  Q  Not  sure  Explain 


d.  Did  you  have  the  impression  at  any  time  that  any  figures  were  deliberately 
being  "doctored"?  Q  Yes  q  Nq  q  Hofc  __,,  3-^ 


e.  Did  you  have  the  impression  at  any  time  that  any  figures  were  being  withheld 
or  suppressed?  Dyes  Q  No  DNat  suxe      Expiain_____ 


f .  Did  you  leave  an  interview  evaluation  form  with  the  panel  member? 

□  Yes  []No  DNot  applicable 
What  was  the  respondent's  reaction? 


Appendix  C  107 


k.     SUMMARY  COMMENTS 

Briefly  summarize  the  respondent's  over-all  reaction  to  the  interview.  Please 
include  any  information  which  you  feel  may  he  helpful  in  understanding  and 
analyzing  the  data  of  this  panel  member. 


5.  TNTERVTEWSR fS  NOTES 

This  IRF  will  he  returned  to  you  on  the  next  wave  so  use  this  space  to  record 
any  information  you  think  you  may  need.  Following  are  suggestions. 

Future  contact  arrangements 

Who 

When 


Where_ 
How 


Any  information  which  will  make  your  interview  on  the  next  wave  easier,  touchy 
data  areas  or  subjects,  hobbies  or  other  talking  points,  personal  likes  or 
dislikes,  etc. 
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